Automating .NET Framework deployments with AWS CodePipeline to Elastic Beanstalk
When it comes to Windows CI/CD pipeline people immediately start thinking about tools like Jenkins, Octopus, or Azure DevOps, and don’t get me wrong because those are still great tools to deal with CI/CD complexities. However, today I will be explaining how to implement a simpler .NET Framework (Windows) CI/CD pipeline that will deploy two […]
Data Archiving utilising Managed Workflows for Apache Airflow
We assisted a Fintech client to minimize its storage cost by archiving its data from RDS (MySQL) to S3 using an automated batch process, where all data from a specific time range should be exported to S3. Once the data is stored on S3 the historical data can be analyzed using AWS Athena and Databricks. […]
What is the Real Cost of a Data Breach in 2022?
Did Data Breaches increase in 2021? One of the biggest changes that occurred as a result of the COVID-19 pandemic is the way in which we work. Whilst remote work began as a temporary fix to deal with lockdowns, it is a shift that has been embraced by numerous businesses over the past two years. […]
Data Dependency
The Importance of Data Dependency Why not investing in data platforms is setting your company up for disaster. Companies with legacy systems or workloads face one of three problems more often than not. Maybe your company has already experienced issues with time to market, bugs in production or limited task coverage due to a lack […]
Using DbT and Redshift to provide efficient Quicksight reports
TL;DR: Using Redshift as a Data Warehouse to integrate data from AWS Pinpoint, AWS DynamoDB, Microsoft Dynamics 365 and other external sources. Once the data is ingested to Redshift, DbT is used to transform the data into a format that is easier to be consumed by AWS Quicksight. Each Quicksight report/chart has a fact table. […]
Launching Amazon FSx for Windows File Server and Joining a Self-managed Domain using Terraform
TL;DR:The github repo with all scripts are here. Because of specific requirements, reasons, or preferences, some customers need to self-manage a Microsoft AD directory on-premises or in the cloud. AWS offers options to have their fully managed Microsoft Windows file servers (Amazon FSx for Windows File Server) join a self-managed Microsoft Active Directory. In this […]
AWS OpenTelemetry for centralised observability in a multi-account architecture
As a cloud consulting company, we at DNX work directly with many Fintechs and SaaS companies using AWS. Many of these companies need to meet high levels of compliance providing their software in a single-tenant architecture, where each of their customers has their own AWS account to guarantee full isolation between customers. This raises a […]
How to deploy an ALB + ASG + EC2 using AWS CDK and TypeScript
Have you heard about Cloud Development Kit or CDK? [Yes, No] What is AWS CDK? The AWS Cloud Development Kit (CDK), lets you define your cloud Infrastructure as Code (IaC) in one of five supported programming languages. It is intended for moderately to highly experienced AWS users. In this blog post, you will see how […]
Using taints and tolerations in EKS to run machine learning pods on GPU nodes
One of our clients uses EKS and they had a problem: running machine learning pods on a special Kubernetes node group that has GPU nodes. We solved this problem using Kubernetes taints, tolerations, labels, and node selectors. This is how to configure it: Choose an Amazon EKS-optimized Arm Amazon Linux AMI and install the NVIDIA […]
How to use Apache Airflow CLI with Amazon MWAA
Photo by John Schnoon on Unsplash Amazon MWAA (Managed Workflow for Apache Airflow) was released by AWS at the end of 2020. This brand new service provides a managed solution to deploy Apache Airflow in the cloud, making it easy to build and manage data processing workflows in AWS. MWAA enables automatic deployment of all the infrastructure […]