Get To Know Google Pub-Sub And Its Message Queue
Global messaging and easier absorption of events. Absorbing events on all scales Data absorption is the foundation for analysis and machine learning, whether you are creating integrated streams, batches, or pipelines. Google cloud Pub-Sub / Sub provides a simple and reliable staging location for your event data for processing, storage, and analysis purposes.
With Cloud Pub / Sub, data engineers can:
1. Scale by providing, creating partitions, or isolating loads without worry
2. Easily expand your application and pipeline to new regions with global topics
3. Enrich, delete duplicates, sort, merge, and place events using Cloud Dataflow
4. Combining real-time processing and batch processing through durable Cloud Pub / Sub storage
Simplify the development of event-controlled microservices
Whether you’re just starting an asynchronous event-driven microservice activity or migrating an existing system, making events accessible via messaging middleware is an important first step. Application developers at GCP rely on Cloud Pub / Sub to send each event reliably to all services that will respond to it.
After publishing events to Cloud Pub / Sub:
1. A push subscription sends events to applications without a server running on Cloud Functions, App Engine or Cloud Run
2. Pull subscriptions provide it to more complex stateful services that run on Google Kubernetes Engine or Cloud Dataflow
3. The multi-region environment operates smoothly thanks to the global Cloud Pub / Sub
Be ready for production from day one
Cloud Pub / Sub is designed as a premium service that allows Google Cloud users to focus on application logic, regardless of location or scale. This service is very minimal and easy to start, but it also eliminates operational, scaling, compliance, and security shocks that are revealed in the software project themselves.
That is why Cloud Pub / Sub includes the following active features:
1. Comprehensive encryption, IAM, and audit logging
2. Scaling and supplying fully automatic and without operation with almost unlimited throughput
3. Extreme data security and availability with synchronized cross-zone replication
4. The native client library in several major languages and open service APIs
Cloud Pub / Sub features
1. Delivery of at least one time
Synchronous cross-zone message replication and receipt tracking per message ensure delivery at least once on all scales.
2. Open it
Open client APIs and libraries in seven languages support cross-cloud and hybrid deployments.
3. Processing exactly once
Cloud Dataflow supports reliable, expressive, and one-time processing of Cloud Pub / Sub streams.
4. Global by default
Publish from anywhere in the world and use from anywhere, with consistent latency. No replication needed.
5. Without the provision, everything is done automatically
Cloud Pub / Sub has no shards or partitions. Just set your quota, publish, and use.
6. Compliance and safety
Cloud Pub / Sub is a service that complies with HIPAA provisions and offers very detailed access control and full encryption.
Take advantage of integration with several services, such as cloud storage and Gmail update events, and Cloud Functions for event-controlled, server-free computing.
8. Search and replay
Rewind your backlog to any point in time or snapshot, giving it the ability to reprocess messages. Play fast forward to get rid of obsolete data.
Cloud Pub / Sub prices
The more you use Cloud Pub / Sub, the cheaper the price will be. No upfront costs and no fees for creating or managing topics or subscriptions.
|Monthly data volume 1||Price per TB 2|
|First 10 GB||$ 0.00|
|Next 50 TB||$ 60|
|Next 100 TB||$ 50|
|Above 150 TB||$ 40|
Google Cloud Pub / Sub is a message queue service that allows systems to share data with other systems in real-time. To do this, the following things are needed.
1. First is Topic, which is a feed where data will be published
2. Second is Publisher, the process of publishing data to a topic, and
3. Finally, a Subscriber is a system that will receive data from topics sent from the publisher.
Google Cloud Platform products used are Google Cloud Shell is a shell environment, Google Cloud Pub / Sub as a messaging service, and Python as a Programming language.
Next is the system architecture that will be created. To do the above, do the following:
1. install the google cloud pub subpackage on the google cloud shell.
2. set GOOGLE_CLOUD_PROJECT variable environment.
3. Make a publisher.py file that contains.
This file contains methods for creating topics, topic lists, and publishing messages
1. Create a Pub / Sub Topic with the name of the song.
2. List topics to ensure topics have been created.
3. Create a file called subscriber.py to receive messages from the publisher.
In this script, there are methods to create a subscriber from a topic, subscriber list on a topic, and receive messages from a topic
1. Create a subscriber with the name subs_lagu to retrieve messages from the song topic.
2. List subscriber that receives from the song topic.
3. Listen to messages from song topics and subscriber subs_lagu.
4. Add a session to the cloud shell.
5. Push message to the song topic by the publisher using the new session so that it does not terminate the session that is listening to the topic.
6. Look at the session that is listening to the message from the song topic. Then it will produce output like this.
Demo of a program that has been created.