Get To Know Google Pub-Sub And Its Message Queue

Get To Know Google Pub Sub And Its Message Queue

Get To Know Google Pub-Sub And Its Message Queue

Cloud Pub / Sub

Global messaging and easier absorption of events. Absorbing events on all scales Data absorption is the foundation for analysis and machine learning, whether you are creating integrated streams, batches, or pipelines. Google cloud Pub-Sub / Sub provides a simple and reliable staging location for your event data for processing, storage, and analysis purposes.

Get To Know Google Pub Sub And Its Message Queue
Get To Know Google Pub Sub And Its Message Queue

With Cloud Pub / Sub, data engineers can:

1. Scale by providing, creating partitions, or isolating loads without worry
2. Easily expand your application and pipeline to new regions with global topics
3. Enrich, delete duplicates, sort, merge, and place events using Cloud Dataflow
4. Combining real-time processing and batch processing through durable Cloud Pub / Sub storage

Simplify the development of event-controlled microservices

Whether you’re just starting an asynchronous event-driven microservice activity or migrating an existing system, making events accessible via messaging middleware is an important first step. Application developers at GCP rely on Cloud Pub / Sub to send each event reliably to all services that will respond to it.

After publishing events to Cloud Pub / Sub:

1. A push subscription sends events to applications without a server running on Cloud Functions, App Engine or Cloud Run
2. Pull subscriptions provide it to more complex stateful services that run on Google Kubernetes Engine or Cloud Dataflow
3. The multi-region environment operates smoothly thanks to the global Cloud Pub / Sub

Be ready for production from day one

Cloud Pub / Sub is designed as a premium service that allows Google Cloud users to focus on application logic, regardless of location or scale. This service is very minimal and easy to start, but it also eliminates operational, scaling, compliance, and security shocks that are revealed in the software project themselves.

That is why Cloud Pub / Sub includes the following active features:

1. Comprehensive encryption, IAM, and audit logging
2. Scaling and supplying fully automatic and without operation with almost unlimited throughput
3. Extreme data security and availability with synchronized cross-zone replication
4. The native client library in several major languages ​​and open service APIs

Cloud Pub Sub For Global Messaging And Event Absorption
Cloud Pub Sub For Global Messaging And Event Absorption

Cloud Pub / Sub features

1. Delivery of at least one time

Synchronous cross-zone message replication and receipt tracking per message ensure delivery at least once on all scales.

2. Open it

Open client APIs and libraries in seven languages ​​support cross-cloud and hybrid deployments.

3. Processing exactly once

Cloud Dataflow supports reliable, expressive, and one-time processing of Cloud Pub / Sub streams.

4. Global by default

Publish from anywhere in the world and use from anywhere, with consistent latency. No replication needed.

5. Without the provision, everything is done automatically

Cloud Pub / Sub has no shards or partitions. Just set your quota, publish, and use.

6. Compliance and safety

Cloud Pub / Sub is a service that complies with HIPAA provisions and offers very detailed access control and full encryption.

7. Integrated

Take advantage of integration with several services, such as cloud storage and Gmail update events, and Cloud Functions for event-controlled, server-free computing.

8. Search and replay

Rewind your backlog to any point in time or snapshot, giving it the ability to reprocess messages. Play fast forward to get rid of obsolete data.

Cloud Pub / Sub prices

The more you use Cloud Pub / Sub, the cheaper the price will be. No upfront costs and no fees for creating or managing topics or subscriptions.

Monthly data volume 1 Price per TB 2
First 10 GB $ 0.00
Next 50 TB $ 60
Next 100 TB $ 50
Above 150 TB $ 40

Google Cloud Pub / Sub is a message queue service that allows systems to share data with other systems in real-time. To do this, the following things are needed.

1. First is Topic, which is a feed where data will be published
2. Second is Publisher, the process of publishing data to a topic, and
3. Finally, a Subscriber is a system that will receive data from topics sent from the publisher.

Google Cloud Platform products used are Google Cloud Shell is a shell environment, Google Cloud Pub / Sub as a messaging service, and Python as a Programming language.

Google Pub Sub As A Messaging Service, And Python As A Programming Language
Google Pub Sub As A Messaging Service, And Python As A Programming Language

Next is the system architecture that will be created. To do the above, do the following:

1. install the google cloud pub subpackage on the google cloud shell.

sudo pip install – upgrade google-cloud-pubsub

2. set GOOGLE_CLOUD_PROJECT variable environment.

export GLOBAL_CLOUD_PROJECT = <GCP-Project-ID>

3. Make a publisher.py file that contains.

This file contains methods for creating topics, topic lists, and publishing messages

1. Create a Pub / Sub Topic with the name of the song.

python publisher.py $ GLOBAL_CLOUD_PROJECT create a song

2. List topics to ensure topics have been created.

python publisher.py $ GLOBAL_CLOUD_PROJECT list

3. Create a file called subscriber.py to receive messages from the publisher.

In this script, there are methods to create a subscriber from a topic, subscriber list on a topic, and receive messages from a topic

1. Create a subscriber with the name subs_lagu to retrieve messages from the song topic.

python subscriber.py $ GLOBAL_CLOUD_PROJECT create song subs_song

2. List subscriber that receives from the song topic.

python subscriber.py $ GLOBAL_CLOUD_PROJECT list_in_topic song

3. Listen to messages from song topics and subscriber subs_lagu.

python subscriber.py $ GLOBAL_CLOUD_PROJECT receive subs_song

4. Add a session to the cloud shell.

5. Push message to the song topic by the publisher using the new session so that it does not terminate the session that is listening to the topic.

python publisher.py $ GLOBAL_CLOUD_PROJECT publish the song “upset night of the week”

6. Look at the session that is listening to the message from the song topic. Then it will produce output like this.

Demo of a program that has been created.

Author: Marru

I am on a mission to help empower young people and help people get to know technology using it optimally.