Сценарий powershell для очистки всех очередей rabbit mq для всех vhosts

Introduction​

Prerequisites

This tutorial assumes RabbitMQ is installed, running on
localhost and the stream plugin enabled.
The standard stream port is 5552. In case you
use a different host, port or credentials, connections settings would require
adjusting.

Using docker

If you don’t have RabbitMQ installed, you can run it in a Docker container:


'-rabbitmq_stream advertised_host localhost'

wait for the server to start and then enable the stream and stream management plugins:


Where to get help

“Hello World”​

(using the .NET/C# Stream Client)

In this part of the tutorial we’ll write two programs in C#; a
producer that sends a single message, and a consumer that receives
messages and prints them out. We’ll gloss over some of the detail in
the .NET client API, concentrating on this very simple thing just to get
started. It’s the “Hello World” of RabbitMQ Streams.

The .NET stream client library

RabbitMQ speaks multiple protocols. This tutorial uses RabbitMQ stream protocol which is a dedicated
protocol for RabbitMQ streams. There are a number of clients
for RabbitMQ in many different
languages
, see the stream client libraries for each language.
We’ll use the .NET stream client provided by RabbitMQ.

The client supports .NET.
This tutorial will use RabbitMQ .NET stream client 1.8.0 and .NET, so you make sure
you have it installed and in your PATH.

RabbitMQ .NET stream client 1.8 and later versions are distributed via nuget.

This tutorial assumes you are using PowerShell on Windows. On MacOS and Linux nearly
any shell will work.

Setup

First let’s verify that you have the .NET toolchain in PATH:

Running that command should produce a help message.

An executable version of this tutorial can be found in the RabbitMQ tutorials repository.

Now let’s generate two projects, one for the publisher and one for the consumer:

dotnet new console 

dotnet new console

This will create two new directories named Send and Receive.

Then we add the client dependency.

dotnet add package RabbitMQ
dotnet add package RabbitMQ

Now we have the .NET project set up we can write some code.

Sending

We’ll call our message producer (sender) Send.cs and our message consumer (receiver)
Receive.cs. The producer will connect to RabbitMQ, send a single message,
then exit.

In
Send.cs,
we need to use some namespaces:

then we can create a connection to the server:


The entry point of the stream .NET client is the StreamSystem.
It is used for configuration of RabbitMQ stream publishers, stream consumers, and streams themselves.

It abstracts the socket connection, and takes care of protocol version negotiation and authentication and so on for us.

This tutorial assumes that stream publisher and consumer connect to a RabbitMQ node running locally, that is, on localhost.
To connect to a node on a different machine, simply specify target hostname or IP address on the StreamSystemConfig.

Next let’s create a producer.

The producer will also declare a stream it will publish messages to and then publish a message:





The stream declaration operation is idempotent: the stream will only be created if it doesn’t exist already.

A stream is an append-only log abstraction that allows for repeated consumption of messages until they expire.
It is a good practice to always define the retention policy.
In the example above, the stream is limited to be 5 GiB in size.

The message content is a byte array.
Applications can encode the data they need to transfer using any appropriate format such as JSON, MessagePack, and so on.

When the code above finishes running, the producer connection and stream-system
connection will be closed. That’s it for our producer.

Each time the producer is run, it will send a single message to the server and the message will be appended to the stream.

The complete Send.cs file can be found on GitHub.

Sending doesn’t work!

If this is your first time using RabbitMQ and you don’t see the “Sent”
message then you may be left scratching your head wondering what could
be wrong. Maybe the broker was started without enough free disk space
(by default it needs at least 50 MB free) and is therefore refusing to
accept messages. Check the broker log file to see if there
is a resource alarm logged and reduce the
free disk space threshold if necessary.
The Configuration guide
will show you how to set disk_free_limit.

Receiving

The other part of this tutorial, the consumer, will connect to a RabbitMQ node and wait for messages to be pushed to it.
Unlike the producer, which in this tutorial publishes a single message and stops, the consumer will be running continuously, consume the messages RabbitMQ will push to it, and print the received payloads out.

Similarly to Send.cs, Receive.cs will need to use some namespaces:

When it comes to the initial setup, the consumer part is very similar the producer one; we use the default connection settings and declare the stream from which the consumer will consume.

:/>  Xbox wireless adapter for windows установка заблокирована групповой политикой

Note that the stream name must match that used by the producer.




Note that the consumer part also declares the stream.
This is to allow either part to be started first, be it the producer or the consumer.

The Consumer class is used to instantiate a stream consumer and the ConsumerConfig record to configure it.
We provide a MessageHandler callback to process delivered messages.

The OffsetSpec property defines the starting point of the consumer.
In this case, the consumer starts from the very first message available in the stream.




The complete Receive.cs file can be found on GitHub.

Putting It All Together

In order to run both examples, open two terminal (shell) tabs.

Both parts of this tutorial can be run in any order, as they both declare the stream.
Let’s run the consumer first so that when the first publisher is started, the consumer will print it:

Then run the producer:

The consumer will print the message it gets from the publisher via
RabbitMQ. The consumer will keep running, waiting for new deliveries. Try re-running
the publisher several times to observe that.

Streams are different from queues in that they are append-only logs of messages
that can be consumed repeatedly.
When multiple consumers consume from a stream, they will start from the first available message.

Introduction​

Prerequisites

This tutorial assumes RabbitMQ is installed, running on
localhost and the stream plugin enabled.
The standard stream port is 5552. In case you
use a different host, port or credentials, connections settings would require
adjusting.

Using docker

If you don’t have RabbitMQ installed, you can run it in a Docker container:


'-rabbitmq_stream advertised_host localhost'

wait for the server to start and then enable the stream and stream management plugins:


Where to get help

“Hello World”​

(using the Node.js Stream Client)

In this part of the tutorial we’ll write two programs in JavaScript; a
producer that sends a single message, and a consumer that receives
messages and prints them out. We’ll gloss over some of the detail in
the JavaScript client API, concentrating on this very simple thing just to get
started. It’s the “Hello World” of RabbitMQ Streams.

The Node.js stream client library

RabbitMQ speaks multiple protocols. This tutorial uses RabbitMQ stream protocol which is a dedicated
protocol for RabbitMQ streams. There are a number of clients
for RabbitMQ in many different
languages
, see the stream client libraries for each language.
We’ll use the Node.js stream client built and supported by Coders51.

The client supports Node.js >= 16.x.
This tutorial will use Node.js stream client 0.3.1 version.
The client 0.3.1 and later versions are distributed via npm.

This tutorial assumes you are using powershell on Windows. On MacOS and Linux nearly
any shell will work.

Setup

First let’s verify that you have the Node.js toolchain in PATH:

Running that command should produce a help message.

Now let’s create a project:

then install the client:


This is how package.json should look like:

"Tutorial for the nodejs RabbitMQ stream client"


Now create new files named receive.js and send.js.
Now we have the Node.js project set up we can write some code.

Sending

We’ll call our message producer (sender) send.js and our message consumer (receiver)
receive.js. The producer will connect to RabbitMQ, send a single message,
then exit.

In
send.js,
we need to add the client:


then we can create a connection to the server:

The entry point of the client is the Client class.
It is used for stream management and the creation of publisher instances.

It abstracts the socket connection, and takes care of protocol version negotiation and authentication and so on for us.

This tutorial assumes that stream publisher and consumer connect to a RabbitMQ node running locally, that is, on localhost.
To connect to a node on a different machine, simply specify target hostname or IP address Client parameters.

Next let’s create a producer.

The producer will also declare a stream it will publish messages to and then publish a message:




The stream declaration operation is idempotent: the stream will only be created if it doesn’t exist already.

A stream is an append-only log abstraction that allows for repeated consumption of messages until they expire.
It is a good practice to always define the retention policy.
In the example above, the stream is limited to be 5 GiB in size.

The message content is a byte array.
Applications can encode the data they need to transfer using any appropriate format such as JSON, MessagePack, and so on.

:/>  Какие клавиши нажать чтобы сохранить файл

When the code above finishes running, the producer connection and stream-system
connection will be closed. That’s it for our producer.

Each time the producer is run, it will send a single message to the server and the message will be appended to the stream.

The complete send.js file can be found on GitHub.

Sending doesn’t work!

If this is your first time using RabbitMQ and you don’t see the “Sent”
message then you may be left scratching your head wondering what could
be wrong. Maybe the broker was started without enough free disk space
(by default it needs at least 50 MB free) and is therefore refusing to
accept messages. Check the broker log file to see if there
is a resource alarm logged and reduce the
free disk space threshold if necessary.
The Configuration guide
will show you how to set disk_free_limit.

Receiving

The other part of this tutorial, the consumer, will connect to a RabbitMQ node and wait for messages to be pushed to it.
Unlike the producer, which in this tutorial publishes a single message and stops, the consumer will be running continuously, consume the messages RabbitMQ will push to it, and print the received payloads out.

Similarly to send.js, receive.js will need to use the client:


When it comes to the initial setup, the consumer part is very similar the producer one; we use the default connection settings and declare the stream from which the consumer will consume.


Note that the consumer part also declares the stream.
This is to allow either part to be started first, be it the producer or the consumer.

We use the declareConsumer method to create the consumer.
We provide a callback to process delivered messages.

offset defines the starting point of the consumer.
In this case, the consumer starts from the very first message available in the stream.

The complete receive.js file can be found on GitHub.

Putting It All Together

In order to run both examples, open two terminal (shell) tabs.

Both parts of this tutorial can be run in any order, as they both declare the stream.
Let’s run the consumer first so that when the first publisher is started, the consumer will print it:

Then run the producer:

The consumer will print the message it gets from the publisher via
RabbitMQ. The consumer will keep running, waiting for new deliveries. Try re-running
the publisher several times to observe that.

Streams are different from queues in that they are append-only logs of messages
that can be consumed repeatedly.
When multiple consumers consume from a stream, they will start from the first available message.

I’m runnings tests thath fills RabbitMQ queues with messages. I have multiple vhosts and each host has multiple queues with lots of messages. I want a Powershell script that will simply purge all messages from all queues for all vhosts. There is already a script for Python that purges all queues for a single vhost, but I want to use powershell and I want to purge all queues for all vhosts.

I don’t want to delete the queues or vhosts, I only want to purge the queues of all messages. I’m using RabbitMQ version 3.12.10.

Does anyone have such a script?

FluffyBike's user avatar

I created a powershell script based on the ‘rabbitmqctl’ documentation. The script will only delete unacked messages.
The script uses 3 rabbitmqctl commands:

  1. rabbitmqctl list_vhosts > vhosts.txt Which fetches a list of all vhosts and stores the list to a file.
  2. rabbitmqctl list_queues -p $vhost > queues.txt Which fetches a list of all queues for a given vhost, and stores the queue names to a file.
  3. rabbitmqctl purge_queue -p $vhost $queueName Which purges a given queue for a give vhost.

Remember to add the RabbitMQ bin dir to you environment Path variable to enable rabbitmqctl in the powershell terminal. The script is filtering out some of the generic progam output, so based on what version of RabbitMQ you are using and you environment, you may have to modify the script to work for you.

Here’s the script:

rabbitmqctl list_vhosts > vhosts.txt
foreach($vhost in Get-Content .\vhosts.txt) { if ($vhost.StartsWith("Listing vhosts") -or $vhost.StartsWith("name")) { # skipping the generic program output continue; } Write-Host "Checking vhost $($vhost) for queues" rabbitmqctl list_queues -p $vhost > queues.txt foreach ($queueLine in Get-Content .\queues.txt) { if ($queueLine.StartsWith("Timeout: ") -or $queueLine.StartsWith("Listing queues") -or $queueLine.StartsWith("name")) { # skipping the generic program output continue; } $split = $queueLine -split "\s+" # splitting string on both space and tab $queueName = $split[0] $messageCount = $split[1] if ($messageCount -eq 0 ) { continue; } rabbitmqctl purge_queue -p $vhost $queueName }
}
Remove-Item "vhosts.txt"
Remove-Item "queues.txt"

FluffyBike's user avatar

3 gold badges13 silver badges26 bronze badges

Update: Finally I’ve found where is the problem. It is RabbitMQ 3.1.17.

Recently upgraded my “Kitchen laptop” used for some “breakfast experiments” to the latest LabVIEW, and the problem coming back (or still here).

:/>  Victoria HDD 4.47 и 5.37 Rus версия

The problem is that this laptop is not very powerful, just i7-3740QM and every time when such spike happened, the СPU’s Fan is turned on shortly, then goes off. Every 5-6 seconds, very nervous.

Screenshot 2024-02-11 07.54.50.png

So, the solution is to place missing handle.exe from SysInternals to the folder “C:\ProgramData\National Instruments\Skyline\RabbitMQ”, this will make Erlang and RabbitMQ happy and no CPU’s spikes any longer, and laptop now silent back, and I’m happy as well.

Screenshot 2024-02-11 07.37.02.png

This is general philosophical engineering problem of the contintiously growing and complicated systems, when one product depends on other product and some day everything goes out of the control and nobody knows how to fix it.

This documentation provides information about the different types of RabbitMQ and Helper installations, the usage of a Site Connector, and the validation for the RabbitMQ product.

Software Downloads and Requirements

To view the compatible versions of the RabbitMQ-related applications, see the Compatibility Matrix

RabbitMQ Helper Software Downloads
Software and Download LinksAdditional Notes
RabbitMQ Helper (current version)

To install RabbitMQ Helper for version 10.5.0.0 and above:

  1. Run on cmd: cd <helper-installer-directory>.

  2. Run on cmd : msiexec /iDelinea.RabbitMq.Helper.xx.x.x.x.msi /l*v install.log.

Microsoft.NET – Windows Server Hosting (v7.0 or above)We recommend installing .NET before you install .
PowerShell Core (v7.3.2 or above)

Install PowerShell 7 before performing the installation. To install the PowerShell Core:

  1. Run the installer.

  2. Accept the terms in the license agreement.

  3. Click Install and Finish.

ErLang (current version) 
RabbitMQ Server (current version) 

Installing RabbitMQ and Setting Up a Site Connector

A Site Connector is a combination of Erlang and RabbitMQ. To view the installation requirements for setting up a site connector, open and navigate to Installing RabbitMQ > Installation.

Downloading Installers using RabbitMQ Helper

 -Verbose 
-Verbose 

Alternatively you can download the installers using official CND mirrors:

 -UseNonMirror -Verbose 
–UseNonMirror -Verbose

Installing RabbitMQ Helper

To install for v10.5.0.0 and above:

  1. Switch to the directory where the .msi file exists using: cd <helper-msi-directory-path>

  2. Run the msiexec /i Delinea.RabbitMq.Helper.10.6.0.0.msi /l*v install.log command.

    Change the name of the .msi file as per the requirements outlined for your version.

To install for v10.4.2.0 and below:

  1. Run the executable from the installation folder.

    1. PROGRAMFILES%\Delinea Software Ltd\RabbitMQ Helper\net6.0\Delinea.RabbitMQ.Helper.exe

  2. This program prepares and runs a Windows PowerShell instance that is pre-configured to use the PowerShell module.

Installing RabbitMQ

  • Simple installations (without TLS):

  • Advanced installations (with TLS):

After you complete the installation, will open a browser window loading the RabbitMQ management console. You can close that window for now.

Validating the RabbitMQ Installation

After successfully installing RabbitMQ, the next step is to validate the installation.

  1. Assert-RabbitMqConnectivity—Validates the connectivity to RabbitMQ.

  2. Assert-RabbitIsRunning—Validates that RabbitMQ is running on the current host.

About the Install-Connector Cmdlet

The table below lists the commands to process in the Install-Connector cmdlet. Use the Install-Connector cmdlet to install RabbitMQ and not the individual commands available.

Install-Connector Cmdlet commands
CommandDescription
Get-ErlangInstallerDownloads the currently supported Erlang installer.
Get-RabbitMqInstallerDownloads the currently supported RabbitMQ installer.
Uninstall-RabbitMq

Uninstalls the previous RabbitMQ installation.

Uninstall-ErlangUninstalls the previous Erlang installation.
Set-ErlangHomeEnvironmentalVariableSets the Erlang and RabbitMQ environmental variables.
Install-ErlangInstalls Erlang.
New-RabbitMqConfigDirectoryCreates a custom RabbitMQ configuration directory.
Set-RabbitMqBaseEnvironmentalVariable
  • Sets the RabbitMQ BASE environment variable to the created configuration directory.
  • With TLS
    Convert-CaCerToPemConverts a CA certificate to the PEM file format.
    Convert-PfxToPemConverts a Host PFX to the PEM file format.
    Convert-CngOrEccToPemConverts a CNG or ECC key type certificate to the PEM file format.
    New-RabbitMqTlsConfigFilesCreates a RabbitMQ TLS configuration file.
    Install-RabbitMqInstalls RabbitMQ.
    Copy-ErlangCookieFileCopies the Erlang system cookie to the current user profile.
    Assert-RabbitMqIsRunningVerifies that RabbitMQ is running.
    Enable-RabbitMqManagementEnables the RabbitMQ management UI.
    New-RabbitMqUserCreates a basic user.
    Grant-RabbitMqUserPermissionGrants permissions to the created user.
    Assert-RabbitMqConnectivityVerifies that the newly created user can connect to RabbitMQ with TLS.
    Delete-RabbitMqUser
  • Deletes a default Admin(guest) in RabbitMQ.
  • Create RabbitMQ Management AdminCreates an Admin User for RabbitMQ Management UI access and for running commands that need Admin access.
    Without TLS
    New-RabbitMqNonTlsConfigFilesCreates a RabbitMQ non-TLS configuration file.
    Install-RabbitMqInstalls RabbitMQ.
    Copy-ErlangCookieFileCopies the Erlang system cookie to the current user profile.
    Assert-RabbitMqIsRunningVerifies that RabbitMQ is running.
    Enable-RabbitMqManagementEnables the RabbitMQ management UI.
    New-RabbitMqUserCreates a basic user.
    Grant-RabbitMqUserPermissionGrants permissions to the created user.
    Assert-RabbitMqConnectivity
    Delete-RabbitMqUserDeletes a default Admin(guest) in RabbitMQ.
    Create RabbitMQ Management AdminCreates an Admin User for RabbitMQ Management UI access and for running commands that need Admin access.