kaliop/queueingbundle-sqs

Kaliop Queueing Bundle - AMAZON SQS plugin

Kaliop Queueing Bundle - AMAZON SQS plugin

Adds support for AWS Simple Queueing Service to the Kaliop Queueing Bundle, _(*1)

See: http://aws.amazon.com/sqs/ and https://github.com/kaliop-uk/kueueingbundle respectively., _(*2)

It has been given its own bundle because it has higher requirements than the base Queueing Bundle, _(*3)

Installation

1. Install the bundle via Composer., _(*4)

2. Enable the KaliopQueueingPluginsSQSBundle bundle in your kernel class registerBundles()., _(*5)

3. Clear all caches if not on a dev environment, _(*6)

Usage

1. If you do not have an AWS account, sign up for one at http://aws.amazon.com/, _(*7)

2. Create an SQS queue, using the web interface: https://console.aws.amazon.com/sqs/home, _(*8)

3. Set up configuration according to your AWS account, _(*9)

   • edit parameters.yml in this bundle

4. check that you can list the queue:, _(*10)

   php app/console kaliop_queueing:managequeue list -isqs
   
   php app/console kaliop_queueing:managequeue info -isqs <queue>

5. push a message to the queue, _(*11)

   php app/console kaliop_queueing:queuemessage -isqs <queue> <jsonpayload>

6. receive messages from the queue, _(*12)

   php app/console kaliop_queueing:consumer -isqs <queue>

Running tests

If you want to run the test suite outside of Travis, you will need to, _(*13)

1. have an AWS SQS account, _(*14)

2. set the following environment variables: SYMFONY__SQS__KEY SYMFONY__SQS__SECRET (note that the test config at the moment hardcodes usage of the us-east-1 region), _(*15)

3. check out somewhere this bundle (no need to install the full Symfony stack), _(*16)

4. run composer install, _(*17)

5. run php vendor/phpunit/phpunit/phpunit Tests/phpunit, _(*18)

Notes

• SQS does not natively support routing-keys the way that RabbitMQ does, nor the exchange/queue topology split. This bundle does add back support for routing-keys, but it is far from ideal; you are encouraged to set up multiple queues instead of using a single queue with multiple consumers which only consumed messages based on routing keys, esp. if you transmit massive amounts of messages in parallel., _(*19)

  The way the bundle supports routing keys is:, _(*20)

  • if the Producer has a routing key set, it will add it to the Message Attributes when sending a Message
  • every Consumer always asks for all messages available in the Queue
  • if the Consumer has a routing key set, and the the message has one in its Message Attributes, the two are matched
  • in case of a match, standard processing goes on: the Consumer sends an ACK call to SQS to signal message reception
  • in case of no match, the Consumer does not send the ACK request to SQS; SQS will then wait for a little while, then put the message back in the queue (the amount of time it waits can be configured per-queue)

  If you find any discrepancy between the way routing keys are matched by RabbitMQ and by this bundle, please report it as a bug., _(*21)

• SQS does not support setting a per-message TTL, only a per-queue one, so all MessageProducers which do have a TTL parameter in their public methods will just ignore it when being used with the SQS driver, _(*22)

• SQS does not guarantee that messages are delivered in the same order they are sent, unless you use FIFO queues. To use a FIFO queue: - create the FIFO queue - in the bundle configuration, set a value for the message_group_id setting for your queue - if you use custom unique message IDs, you will have to set up a service implementing the MessageDeduplicationIdCalculatorInterface interface, and hook it up it using the message_deduplication_id_calculator setting for your queue, _(*23)

• SQS does guarantee that messages are delivered, but it does not guarantee that every message is delivered only once. If such a constraint is important, build unique message IDs in your app and manage them - or use FIFO queues., _(*24)

• For a more in-depth comparison of SQS and RabbitMQ, see f.e. http://blog.turret.io/rabbitmq-vs-amazon-sqs-a-short-comparison/, _(*25)

, _(*26)

, _(*27)