Empty s3 bucket and Delete using Jenkins and AWS CLI
Amazon Simple Storage Service (s3) is a scalable object storage service provided by AWS. It allows users to store and retrieve vast amount of data, make it a fundamental building block for many applications hosted on AWS. Deleting a s3 bucket is a two step process: empty a s3 bucket and then delete a s3 bucket...
Continue reading →
jenkins
aws
Publish to MSK Kafka using AWS Lambda Python
Apache Kafka is a distributed streaming platform designed to handle real-time data streams efficiently. It uses a publish-subscribe model, where producers publish messages to topics, and consumers subscribe to those topics to receive the messages. Amazon MSK simplifies the deployment and management of Kafka clusters on AWS, providing a fully managed and scalable solution. You can use AWS lambda in python to publish to MSK topic...
Continue reading →
kafka
aws
python
Fix SonarQube Error of Public Writable Directories in AWS lambda python
In AWS Lambda functions, it's common to use temporary directories for tasks like generating temporary files, storing intermediate data, or processing data securely. However, improperly managing publicly writable directories can lead to security vulnerabilities. In this article, we'll explore how to address this issue and ensure safe usage of temporary directories in AWS Lambda functions using Python's tempfile module...
Continue reading →
sonarqube
python
aws
Unit Test AWS Lambda Function in Python
AWS lambda function are event driven serverless code. To follow TDD, we should write unit test our lambda functions...
Continue reading →
aws
python
AWS CLI to stop running instances of statemachine
I could not delete a statemachine that had running instances stuck in Progress. Since the number of instances were humongous, it was impossible to manually delete them using UI...
Continue reading →
aws
statemachine
Resolve the Error CROSSSLOT Keys in request don't hash to the same slot
AWS offers managed service Amazon MemoryDB which is redis compatible. The entire keyspace in Redis cluster is divided into hash slots and these slots are assigned to multiple nodes. In redis, getting a single value is straightforward...
Continue reading →
python
redis
memorydb
aws
Cross Region AWS Glue Data Catalog access with Glue ETL
AWS Glue job are fully managed ETL service. Glue job uses hive-compatible metastore called Glue Data catalog.
..
Continue reading →
aws
glue
Create a Lambda Function to run SQL queries in Redshift Cluster
Amazon Redshift is fully managed warehouse service provided by AWS. The Amazon Redshift engine is a SQL-compliant, massively-parallel, query processing and database management system designed to support analytics workload...
Continue reading →
python
sql
lambda function
aws
redshift
Creating Python AWS lambda layer with Docker
When you develop AWS lambda functions, you might feel the need to install additional `python` libraries. This can be achieved using Lambda layers that can be included in any lambda function...
Continue reading →
docker
aws
python
Automate creation of AWS Stack
AWS CDK is a great framework to programmatically deploy cloudformation stack. If you are unfamiliar with AWS CDK, I would recommend first to check out [Getting started with AWS CDK](http://randomwits.com/blog/tutorial-cdk-aws)...
Continue reading →
aws
cdk
cloudformation
typescript
tutorial
Getting Started with AWS CDK
Cloudformation service in AWS allows you to describe an entire set of resources required to make a pipeline. The cloudformation template can be described in JSON or YAML format. Updating the cloudformation template was not a pleasant experience. I wanted to discover alternative ways to generate cloudformation template programmatically...
Continue reading →
aws
cdk
cloudformation
typescript
python
Export DynamoDB Table to S3 Bucket Using Lambda Function
Dynamodb is a great NoSQL service by AWS. One of the most common use case is to export data to s3. There are multiple ways to export data to s3. For this tutorial we will leverage lambda function to achieve our goal...
Continue reading →
aws
lambda function
dynamodb
s3
