Build a Python Flask Azure Queue Storage Producer-Consumer App: Step-by-Step Guide

Here’s a step-by-step guide to creating a producer-consumer application using Python Flask with Azure Queue Storage.

Prerequisites

1. Azure Account: Set up an Azure account if you don’t already have one.
2. Azure Storage Account: Create a storage account in Azure and get the connection string.
3. Azure Queue Storage: Use the Azure Portal to create a queue in the storage account.
4. Environment Setup: Install Python and necessary libraries.

Steps

Step 1: Set Up Python Environment

1. Install Python dependencies:

   pip install flask azure-storage-queue

Step 2: Create Flask Application

1. Create a directory for your project and navigate to it.
2. Create the file structure:

   flask-azure-queue/
   ├── app.py
   ├── producer.py
   ├── consumer.py
   ├── requirements.txt
   

Step 3: Install Dependencies

Add the dependencies to requirements.txt:

flask
azure-storage-queue

Install dependencies:

pip install -r requirements.txt

Step 4: Configure Azure Storage Connection

Create a config.py file for configuration:

# config.py
AZURE_STORAGE_CONNECTION_STRING = "your_connection_string"
QUEUE_NAME = "your-queue-name"

Replace your_connection_string with your Azure Storage connection string and your-queue-name with the name of your queue.

Step 5: Write the Producer Code

Create producer.py to send messages to the Azure Queue:

from azure.storage.queue import QueueClient
from config import AZURE_STORAGE_CONNECTION_STRING, QUEUE_NAME

def send_message(message):
    queue_client = QueueClient.from_connection_string(AZURE_STORAGE_CONNECTION_STRING, QUEUE_NAME)
    queue_client.send_message(message)
    print(f"Message sent: {message}")

Step 6: Write the Consumer Code

Create consumer.py to retrieve and process messages:

from azure.storage.queue import QueueClient
from config import AZURE_STORAGE_CONNECTION_STRING, QUEUE_NAME

def receive_message():
    queue_client = QueueClient.from_connection_string(AZURE_STORAGE_CONNECTION_STRING, QUEUE_NAME)
    messages = queue_client.receive_messages()
    for message in messages:
        print(f"Message received: {message.content}")
        queue_client.delete_message(message)

Step 7: Integrate with Flask

Update app.py to define Flask endpoints:

from flask import Flask, request, jsonify
from producer import send_message
from consumer import receive_message

app = Flask(__name__)

@app.route('/produce', methods=['POST'])
def produce():
    data = request.json
    message = data.get('message')
    if not message:
        return jsonify({'error': 'Message is required'}), 400
    send_message(message)
    return jsonify({'status': 'Message sent'}), 200

@app.route('/consume', methods=['GET'])
def consume():
    receive_message()
    return jsonify({'status': 'Messages processed'}), 200

if __name__ == '__main__':
    app.run(debug=True)

Step 8: Run the Flask Application

1. Start the Flask server:

   python app.py

2. Test the endpoints:
   • To send a message:

     curl -X POST http://127.0.0.1:5000/produce -H "Content-Type: application/json" -d '{"message": "Hello, Azure Queue!"}'
     

   • To process messages:

     curl -X GET http://127.0.0.1:5000/consume

Step 9: Test and Deploy

1. Test your application locally.
2. Deploy it to an environment like Azure App Service, Docker, or any other hosting solution.
3. Ensure the Azure Queue Storage credentials are securely handled in production (e.g., using Azure Key Vault).

This completes the producer-consumer application with Flask and Azure Queue Storage.