Rabbits are M Queuing

So I've been thinking about AI and the next steps towards something interesting... which lead me to streams of consciousness, and the infrastructure of thinking.

Long story short (old-guy speak for TL;DR), this triggered an interest in messaging and streams, specifically Apache Kafka and Rabbit MQ. Today's tale is one of rabbits. It should be noted that my rabbits are still just kits - I'm writing this on day three of my RabbitMQ journey.

Day One Aims

Focus on a repeatable installation of a RabbitMQ cluster, familiarisation with operating the server, sending and receiving messages.

The Overall Result

It was surprisingly easy to install and configure a RabbitMQ Cluster - the scripting is all below. It feels solid.

Follow-up

  • Investigate Federation - perhaps a cluster is not actually needed?
  • Day Two: Do the Tutorials.
  • Day Three: Write up findings.

Environment

I'm running Debian Stable - v12.3 Bookworm. Today is Thursday 14-Dec-2023. I have four hosts: 1 in Australia, 1 in New Zealand and 2 in the UK. The whole install and basic test takes just under 7 minutes, without any effort to parallelise.

Installation Stages

PATH=.:$PATH

10-install-rabbitmq.sh
20-create-cluster.sh
30-create-admin.sh
40-setup-test.sh
50-pump-test.sh

01-installation.sh

Configuration

Tweak this for your setup.

printf "loading config\n"

# whole cluster
hostmq="fogh rogh hinj hern"

# secondary nodes
hostmq2="rogh hinj hern"

# users
adminuser='admin'
#adminpass=`openssl rand -base64 15`
adminpass='qJwt123Y3FJY7X+6OIiG'

auser='alice'
#apass=`openssl rand -base64 15`
apass='A7f1kUsFBz4ynPwJV565'

# todo: set these aliases in user-shells
# ctl - seems to need sudo 
# adm - uses -u user -p password
mqctl='sudo rabbitmqctl'
mqadm='rabbitmqadmin'

# for sudo-scp of secret cookie
ssh_ident='-i /home/jeff/.ssh/id_rsa'

00-config.sh

Install the Software

Went surprisingly well - feels like a solid clean process 👍🏻.

source ./00-config.sh

# define cluster and check network connectivity
for h in $hostmq; do printf "\n$h\n"; ping -c 3 $h; done

# install software
for h in $hostmq; do printf "\n$h\n"
  ssh -tty root@$h 'apt-get -qq update; apt-get install -y rabbitmq-server'
  done

# check the status of the install
for h in $hostmq; do printf "\n$h\n"
  ssh root@$h 'systemctl status rabbitmq-server'
  done

# check the rabbitmq software version
for h in $hostmq; do printf "\n$h\n"
  ssh root@$h 'rabbitmq-diagnostics server_version'
  done
# 3.10.8

10-install-rabbitmq.sh

Create a Cluster

Again, simple and reliable.

source ./00-config.sh

# view installation ownership etc
sudo find /var/lib/rabbitmq/ -ls

# shared secret for cluster
sudo cat /var/lib/rabbitmq/.erlang.cookie

# copy security cookie to all other members of cluster
for h in $hostmq2; do printf "\n$h\ninstalling shared secret\n"
  sudo scp $ssh_ident /var/lib/rabbitmq/.erlang.cookie \
  root@$h:/var/lib/rabbitmq/.erlang.cookie
  
  ssh root@$h \
    'chown rabbitmq:rabbitmq /var/lib/rabbitmq/.erlang.cookie; \
     ls -al /var/lib/rabbitmq/.erlang.cookie; \
     cat /var/lib/rabbitmq/.erlang.cookie '
  done

# restart service so that cookie takes effect
for h in $hostmq2; do printf "\n$h\nrestarting service...\n"
  ssh root@$h 'systemctl restart rabbitmq-server; \
    systemctl status rabbitmq-server'
  done

# configure and start cluster
for h in $hostmq2; do printf "\n$h\njoining cluster...\n"
  ssh root@$h \
    'rabbitmqctl stop_app; \
     rabbitmqctl join_cluster rabbit@fogh; \
     rabbitmqctl start_app'
  done

# check results
$mqctl cluster_status
# Cluster status of node rabbit@fogh ...

20-create-cluster.sh

Create an Admin User

Once this is done you can login to the web interface with the admin credentials.
My dashboard is at http://fogh:15672 - note the port, and be aware it listens on 0.0.0.0!

source ./00-config.sh

# At this stage rabbitmqadmin returns connection refused,
# because management web interface is not enabled

# list available plugins
sudo rabbitmq-plugins list

# enable management
sudo rabbitmq-plugins enable rabbitmq_management

# Note that web interface did not have stats for the other nodes.
for h in $hostmq; \
  do ssh root@$h rabbitmq-plugins enable rabbitmq_management; done

# setup root admin user
$mqctl add_user admin $adminpass
$mqctl set_user_tags admin administrator

# create an admin user with access to everything
# -p      permit?
# /       root vhost
# admin   admin user
# .*      change on all (grant, permissions?)
# .*      write on all
# .*      read on all
$mqctl set_permissions -p / admin ".*" ".*" ".*"

# secure access
$mqctl delete_user guest

# check results
$mqctl list_users
# Listing users ...
# user    tags
# admin   [administrator]

# Admin interface at: http://fogh:15672/
# login using $adminuser / $adminpass

30-create-admin.sh

Setup Test VHost and User

A vhost is like an Apache vhost - it is a context, domain or app within the cluster that will contain exchanges, queues, users etc. I haven't yet found the RabbitMQ Concept Dictionary, so this is inferred, and I'm not yet clear on exchanges and binding 🤷🏻‍♂️.

source ./00-config.sh

# create test-app vhost
$mqctl add_vhost test-app

#$mqctl delete_user $auser

# create alice with password
$mqctl add_user $auser $apass

# make alice an administrator for test-app
$mqctl set_user_tags $auser administrator

# grant alice all permissions for test-app
$mqctl set_permissions $auser --vhost test-app ".*" ".*" ".*"

# check results
$mqctl list_vhosts

$mqctl list_user_permissions alice
# Listing permissions for user "alice" ...
# vhost   configure       write   read
# test-app        .*      .*      .*

40-setup-test.sh

Get Test Pumping

He said: "You need to create exchanges, queues, and bindings for your vhost.". I hope to eventually find basic explanations of what an exchange and a binding is. Queue seems self-explanatory.

source ./00-config.sh

printf "create exchange\n"
# Create new exchange test_exchange
$mqadm -u $auser -p $apass -V test-app \
  declare exchange name=test_exchange type=direct

printf "create queues\n"

# Create quorum queue - queue_type=quorum
$mqadm -u $auser -p $apass -V test-app \
  declare queue name=test_quorum durable=true queue_type=quorum
# queue declared

# Create default classic queue
$mqadm -u $auser -p $apass -V test-app \
  declare queue name=test_classic durable=true
# queue declared

# dont need sudo?
$mqadm -u $auser -p $apass list queues
# +--------------+----------+
# |     name     | messages |
# +--------------+----------+
# | test_classic | 0        |
# | test_quorum  | 0        |
# +--------------+----------+

# create binding for test_quorum
printf "create binding for quorum and classic\n"
$mqadm -u $auser -p $apass -V test-app \
  declare binding \
  source="test_exchange" \
  destination_type="queue" \
  destination="test_quorum" \
  routing_key="test_routing_key_quorum"
# binding declared

# create binding for test_classic
$mqadm -u $auser -p $apass -V test-app \
  declare binding \
  source="test_exchange" \
  destination_type="queue" \
  destination="test_classic" \
  routing_key="test_routing_key_classic"
# binding declared

# publish message for each queue
printf "create messages on quorum\n"
$mqadm -u $auser -p $apass -V test-app \
  publish \
  exchange=test_exchange \
  routing_key=test_routing_key_quorum \
  payload="hello world, Quorum Queue"
# Message published

printf "create messages on classic\n"
$mqadm -u $auser -p $apass -V test-app \
  publish \
  exchange=test_exchange \
  routing_key=test_routing_key_classic \
  payload="hello world, Classic Queue"
# Message published
  
# retrieve quorum messages
printf "\nretrieve quorum on localhost\n"
$mqadm -u $auser -p $apass -V test-app \
  get queue=test_quorum
# hello world, Quorum Queue
    
# retrieve classic messages - this node only
printf "\nretrieve classic on localhost\n"
$mqadm -u $auser -p $apass -V test-app \
  get queue=test_classic
# hello world, Classic Queue

# on remote nodes
for h in $hostmq2; do printf "\nretrieve quorum on $h\n";
  ssh root@$h $mqadm -u $auser -p $apass -V test-app get queue=test_quorum;
  done
# ...

50-pump-test.sh

Now we have to infer concepts and play around with messaging and reliability.

Day Two will be focused on the tutorials.

Jeff, 2023/12/13 05:32

Nuke RabbitMQ

If you want to re-run the whole install. Be careful - there is no confirmation before RabbitMQ is nuked on all hosts!

# nuke software and configuration
for h in $hostmq;
  do printf "\nnuke rabbitmq on $h\n"
  ssh root@$h 'apt-get purge -y rabbitmq-server'
  done

99-nuke-rabbitmq.sh

Credits