RabbitMQ setup and cluster configuration on a windows network


 

Queuing has been integral part of application architecture for quite sometime and therefore there are number of frameworks available to let you manage Queue/Bus.

Queue as you may know is used when you have one node to process your message and Bus (Service Bus/Topic) is used when there is more than one node that needs to process the message. RabbitMQ is one of available framework along with MSMQ, Kafka, Azure Service Bus and etc.

RabbitMQ is one of the most popular framework as it can be used to create highly available and fault tolerant cluster of nodes that hold the queue's and message's for processing.

In the writeup we will try to install RabbitMQ cluster on couple of windows machines which can be easily translated to containerised application using docker.

1) To install docker visit RabbitMQ official and download required file, once downloaded execute the exe and follow the wizard.

You may be asked to install Erlang in the process as that is one of the primary requirement for RabbitMQ setup.

2)Repeat step one for each node that will be part of your cluster (it is recommended to have odd number of nodes/machines).

3)From command prompt go to bin folder of installation directory on each machine which would look like "C:\Program Files\RabbitMQ Server\rabbitmq_server-xx.xx.xx\sbin", and execute below.

rabbitmq-plugins enable rabbitmq_management

This will install admin gui which can be accessed using http://localhost:15672, user name and password is guest.

4)Ensure you are using same erlang cookie on all machines. Cookie is stored in .erlang.cookie and location may vary as per OS for windows 10 it is at C:\Windows\System32\config\systemprofile.

If you have issue location this look at the log file, which is typically located in your user %AppData%\Roaming\RabbitMQ\log directory which would have path corresponding to home dir

I did copy cookie from master node on all other machines

5)Enable below on each machine to allow seamless communication

Rule Type

Program path

Specified local ports

Profile

Name

Port

5672, 15672

Domain, Private, Public

RabbitMQ

Program

%ProgramFiles%\erlx.x.x\bin\erl.exe

Domain, Private, Public

Erlang

Program

%ProgramFiles%\erlx.x.x\erts-a.b.c\bin\erl.exe

Domain, Private, Public

Erlang Run-Time System

Program

%ProgramFiles%\erlx.x.x\erts-a.b.c\bin\epmd.exe

Domain, Private, Public

Erlang Port Mapper Daemon

6)Make sure all machines can communicate with each other if not please add entries in respective host files which can be found at C:\Windows\System32\drivers\etc

7)Identify your master node and execute below on all other machines other than the master node.
rabbitmqctl stop_app
rabbitmqctl join_cluster rabbit@masternodemachinename
rabbitmqctl start_app
rabbitmq-plugins enable rabbitmq_federation
rabbitmq-plugins enable rabbitmq_federation_management

8) After above step you have create a cluster and have all you machined to it, now on any machine if you open rabbmitmq ui using http://localhost:15672 and guest/guest as user name/password you will see that they all are part of a cluster like in below case two machines form a cluster and can be seen in nodes section.



9) Now that we have all nodes added in the cluster time to enable mirroring so that is one node goes down application does not. To configure mirroring execute below.

rabbitmqctl set_policy mirror-all “^config” “{“”ha-mode””:””all””}”

10) To configure load balancing have a look at this

If all goes well you have a cluster of RabbitMQ server which if highly available and fault tolerant.

Comments

Post a Comment

Popular posts from this blog

Gitflow using source tree

Image
In early 2010, Vincent Driessen wrote an article called   “A successful Git branching model”  which recommended an approach called  git-flow  to use git branches in your development cycle. The idea was to standardise branching and merging when developing features, handling releases and managing hot fixes, in order to be consistent and gain the advantages of git’s ‘branchy’ development model. Using many separate branches in Git gives you lots of flexibility, but it can get complex. Adopting a standardised approach has many advantages: ·         Keep your repository tidier ·         Keep your procedures clearer ·         Move between projects more easily with familiar branch structures ·         Get new developers up to speed more quickly SourceTree   now integrates with git-flow and presents it to you in a friendly and intuitive way. Summary of the concept The general idea of git-flow is to use the following branch structure in your repository: ·         Develop
Read more

Component analysis for application security

Image
If you are into writing software you probably follow security practices to keep your application secure. Since most of the platforms are now open source we consume a lot of open source components in our projects. It happened sometime back when a vulnerability was found in one of the popular components(log4j) and companies were clueless about the existence of any such issue as a result many kept running their software on the vulnerable version of log4j. You may not have been impacted or attacked for that matter but never know when it's your turn, is best to have protection against scenarios where components/license used in thrid party component becomes vulnerable/outdated. I recently stumbled upon  Dependency-Track  , it works in two parts:- Identify the list of components in your project, which is also called the SBOM (Software bill of material). CycloneDX can be installed with just a few commands to generate SBOM. Upload BOM to Dependency-Track using UI/API which then scans it aga
Read more

Analyse log using plug and play FileBeat

Image
ELK-B stack is one of the powerful frameworks for log analysis which helps to draw important business decisions and helps evolve ALM. There are few ways to inject data in Elasticsearch for analysis  Writing data using REST API    Using available frameworks(Serilog, NLog, etc) that have an abstraction layer and does all the REST calls behind the scene for you. While changing code is the most popular method but it may not be always possible/viable, also sometimes you do not have access to the environment and all you have is the log files in one format or another.  Filebeat is another powerful plugin in the ELK-B stack that can help you analyze logs without changing code, all you need to do is download FileBeat  and unzip the file. Once you have extracted the content open filebeat.yml and under  filebeat.inputs: of filebeat.yml   locate  paths:          paths maintains list of folders or docker path that will contain logs:-          paths:                   - C:\ProductService\logs\*   
Read more

Introduction to Blazor

Image
WebAssembly (abbreviated Wasm) is a binary instruction format for a stack-based virtual machine. Wasm is designed as a portable compilation target for programming languages, enabling deployment on the web for client and server applications. Blazor is new addition to .Net arsenal which is based on WebAssembly and using that enables developers to create web apps using C# and HTML, hence removing dependency on JavaScript framework like Angular/React and etc. If we are to compare blazor with ASP.NET MVC above would be content in .cshtml file which in this case becomes .razor file, code for MVC controller is also embedded in .blazor page for blazor applications. There are 2 main flavors of Blazor:- 1) Blazor web application. In this case application is hosted in IIS and only html travels to the client application for every action on the page client communicates back to the server with information of action on client side. Server version will then send back response back to the client which
Read more

.NET MAUI and Blazor to create windows/web and mobile application (.NET 6.0)

Image
If you are a .NET developer you would know the following frameworks are generally used for development:- I can tell you from my personal experience that this is not an easy stack to master and something that I have struggled with time. While you can solve problems with the tech stack of windows and the web by using ElectronNET with ASP.NET Core or Blazor, the problem still persists for mobile applications. Now, what if told you that you can do all of the above by just using 1 framework along with Blazor, exciting isn't it. .NET MAUI does just that and saves you from a lot of trouble learning and managing various frameworks. What is .NET MAUI:- .NET Multi-platform App UI (.NET MAUI) is a cross-platform framework for creating native mobile and desktop apps with C# and XAML or Blazor. So if you know Blazor you can write a web app and SPA but when combined with .NET MAUI you can use your skillset to create and deploy the application on web/mobile/desktop. Framework stack now would look
Read more

Clean Architecture

Image
 We have always heard and talked about N-Tier and N-Layer architecture along with SOLID and SOC, but as the size of application along with complexity has changed over a period of time do has application architecture. Also since teams have adopted agile, releases are much more frequent than they used to be a few years back. In the light of the changing landscape, it is important to structure your application design in a way to give best practices, manageable code, loosely coupled features/functionality, and the least possible change propagation into various layers of the application. Popular architecture:- Layered/N-Tiered architecture Monolithic architecture Microservices architecture Event-driven architecture Service-oriented architecture What is Clean Architecture:-     Clean architecture is another name for Onion architecture, it can be viewed as an onion that consists of various application layers, can be primarily divided into 2 parts i.e Core and Peripheral layers. In the below s
Read more

What Interview Is/Is Not for an interviewer

After spending more than decade and having participated in hundreds of interviews, following is my list of what interview "Is" and "Is Not" from interviewers stand point. Interview Is:- Opportunity to meet someone with a set of experience in which they have learned their own ways of doing things and may help you learn something new. To learn what worked for them and what didn't, also the reason why it was success or failure for them. Which provides you free knowledge which can help you avoid your next failure and probably this is not documented and you would have failed. To help build a better work force, so energy invested today in interviewing will save you a lot in future. Interview Is Not:- To ask questions which may not be needed for the role. To ask questions which you can't answer yourself, In doing so you will never know if the person is right/wrong. You can ask such questions to see if they bluff or become impatient but not to test t
Read more

Estimation techniques

Image
Estimating how long it would take to get the job done is not easy, even travelling from point A to point B takes a lot of assumption and risks into consideration. Whether it is ballpark estimation or otherwise you need to use some formal technique, different projects need different kind of estimation and we are going to look into few of them. Lets do ballpark estimation for a website with static pages but requires login. WBS(Work breakdown structure):- In this approach work is broken down to lowest level possible. What remains at the bottom nodes is lost of task which when combined achieves the final goal. In above we have broken down end result into tasks and estimated tasks at various levels, with above in place we now know how many man days are required from various people and plan accordingly. Having an estimate ready helps in better resource and release planning. Its not always possible is recommended to do collaborative estimation(from different teams involve
Read more

WCAG Accessibility

When designing a website or web application we consider other aspects of UI/UX whereas we always forget to consider people with disability. Lets have a look at statistics of users with disability in USA. 19.9 million (8.2%) have difficulty lifting or grasping. This could, for example impact their use of a mouse or keyboard. 15.2 million (6.3%) have a cognitive, mental, or emotional impairment. 8.1 million (3.3%) have a vision impairment. These people might rely on a screen magnifier or a screen reader, or might have a form of color blindness. 7.6 million (3.1%) have a hearing impairment.  They might rely on transcripts and / or captions for audio and video media. If we look at above number large user base is having some sort of disability hence if application is designed with accessibility in mind it can bring in more user which would translate to success of the product/website. Also it is being made mandatory  by law for software applications to be designed as per accessibility standa
Read more