CloudPro

EC2 Image Builder now supports building and testing macOS imagesCloudPro #70: Building Lightweight Kubernetes Dev Ephemeral EnvironmentsOur Exclusive 2-for-1 Sale is LIVE!For the next 24 hours only, you can secure 2 seats for the price of 1 at Generative AI in Action (Nov 11-13)!📅 Sale ends tomorrow at 10 AM ETBring a colleague, friend, or your team and dive into everything this conference has to offer—from expert insights and hands-on sessions to valuable networking opportunities.Act now. This deal won’t last long!⏳Today we will talk about:⭐MasterclassBuilding Lightweight Kubernetes Dev Ephemeral EnvironmentsFrom which Kubernetes pod (and namespace!) is this process that I see on my host?Argo Workflows: Simplify parallel jobs: Container-native workflow engine for KubernetesUsing SimKube 1.0: Comparing Kubernetes Cluster Autoscaler and KarpenterI've joined a company that has an AKS cluster whose version is completely outdated (1.21). I need to upgrade it to version 1.30 without any downtime and have a rollback plan in place🔍Secret KnowledgeLike Heroku, but You Own ItMulti-Metric ScalingGoran Opacic on X: "After years of using @awscloud Aurora, we are moving back to dedicated hardware. MySQL K8s operators are great, storage is cheap, memory is cheap, cpu is cheap, I can run 5.7 as much as I like and no AI. I'll miss database cloning and instant read replicasPolicy as Code in TerraformBehind the scenes of the OpenTelemetry Governance Committee⚡TechwaveEC2 Image Builder now supports building and testing macOS imagesUpgraded Claude 3.5 Sonnet from Anthropic (available now), computer use (public beta), and Claude 3.5 Haiku (coming soon) in Amazon BedrockGrafana 11.3 release: Scenes-powered dashboards, visualization and panel updates, and moreSonar Details OpenAPI Generator Flaw That Creates Source Code VulnerabilityHashiCorp Updates Terraform; Wider Cloud Infrastructure Developer Toolsets🛠️Hackhubkubectl-guard: Accidentally modifying production instead of a local cluster? kubectl-guard helps prevent such critical mistakes.kubesafe: Safely manage multiple Kubernetes clusters by defining safe contexts and protected commands.Tfreveal:An open-source tool that enhances Terraform plan visibility by showing all resource and output differences, including sensitive values.SyncLite:A low-code platform for relational data consolidation, ideal for building data-intensive apps across edge, desktop, and mobile environments.pg_replicateCheers,Shreyans SinghEditor-in-Chief⭐MasterClass: Tutorials & GuidesBuilding Lightweight Kubernetes Dev Ephemeral EnvironmentsKardinal is an open-source tool for creating lightweight, temporary development environments on Kubernetes clusters. It’s designed to minimize resource usage by deploying only the services you need for testing while reusing existing resources when possible. Kardinal introduces “flows”—ephemeral environments that can be spun up for specific features or testing needs, which saves time and costs by avoiding redundant deployments.From which Kubernetes pod (and namespace!) is this process that I see on my host?To find which Kubernetes pod and namespace a process on your host belongs to, you can use crictl along with cgroups. First, get the process ID (PID) of the containerized process, then find its cgroup ID, which will contain the container’s unique identifier. Once you have that ID, use crictl inspect with a formatted output to get the pod’s namespace and name. This retrieves both the namespace and pod name directly from crictl using go-template formatting.Argo Workflows: Simplify parallel jobs: Container-native workflow engine for KubernetesIn this guide, the focus is on Argo Workflows, an open-source tool designed to manage complex workflows in Kubernetes environments by orchestrating parallel tasks in containers. Each step of a workflow is run within a container, making it ideal for complex pipelines like data processing or machine learning. Argo Workflows integrates with Kubernetes services (e.g., volumes, secrets, and RBAC) and uses Directed Acyclic Graphs (DAGs) to sequence tasks. This setup explains deploying Argo on Amazon EKS and integrating it with Argo Events to handle data-driven tasks triggered by messages from Amazon SQS, creating a scalable, event-driven Spark job processing platform on Kubernetes.Using SimKube 1.0: Comparing Kubernetes Cluster Autoscaler and KarpenterSimKube 1.0, a Kubernetes simulator, was used to test two popular cluster autoscaling solutions: Kubernetes Cluster Autoscaler (KCA) and Karpenter. Both tools add nodes to a Kubernetes cluster based on workload demands, but they differ significantly in approach. KCA, originally designed for homogeneous clusters, must be configured with specific instance types, which can make it slower when there are many options. Conversely, Karpenter, designed by AWS, optimizes across all available EC2 instances by default and uses both a "fast" loop for quick scheduling and a "slow" loop for optimization, which made it faster in this simulation.I've joined a company that has an AKS cluster whose version is completely outdated (1.21). I need to upgrade it to version 1.30 without any downtime and have a rollback plan in placeUpgrading an outdated AKS cluster from version 1.21 to 1.30 without downtime requires a careful approach, especially since rolling back AKS upgrades isn't possible. A Blue-Green deployment is a good option here but is complex at the cluster level. One way to approach it is to create a new cluster with AKS version 1.30, deploy and test the application there, and then redirect production traffic to the new cluster via DNS or load balancer once confirmed stable. First, validate the application’s compatibility with version 1.30 in your QA environment and ensure no critical API changes break functionality. If creating a new cluster is challenging due to resource limitations, consider a controlled maintenance window with a staged upgrade (e.g., from 1.21 to 1.22, then to 1.24, and so on) but remember that the direct upgrade might carry risks due to skipped deprecation changes and other breaking updates.🔍Secret Knowledge: Learning ResourcesLike Heroku, but You Own ItDokku is an open-source platform as a service (PaaS) that lets you turn a virtual private server (VPS) into a serverless platform, similar to Heroku, but with more control and no subscription costs. It allows easy deployment of web apps using Docker containers, GitHub Actions, or simple git commands. With features like auto-scaling, built-in SSL from Let’s Encrypt, and password protection, Dokku is ideal for hosting both applications and static sites from private repositories. Additionally, it offers flexible deployment options and can integrate with Cloudflare for HTTPS if needed, making it a powerful, budget-friendly solution for personal or small-scale app hosting.Multi-Metric ScalingYelp has implemented multi-metric autoscaling on its PaaSTA platform, enabling services to scale based on multiple factors (like CPU and request load) rather than just one, improving stability and quicker recovery during high-demand periods. Since PaaSTA is an 11-year-old platform on Kubernetes, updating it safely was challenging. The team spent weeks understanding the codebase, gathering input, and defining a clear, gradual update plan. They used snapshot testing and strict validation to confirm stability at each step, made minimal yet crucial API adjustments, and improved monitoring through Grafana. Ultimately, the update rolled out smoothly, enhancing scaling options without causing any service interruptions.Goran Opacic on X: "After years of using @awscloud Aurora, we are moving back to dedicated hardware. MySQL K8s operators are great, storage is cheap, memory is cheap, cpu is cheap, I can run 5.7 as much as I like and no AI. I'll miss database cloning and instant read replicasPolicy as Code in TerraformPolicy as Code (PaC) allows organizations to enforce rules and guidelines on infrastructure automatically by defining policies as code, ensuring resources meet security, compliance, and operational standards. Tools like HashiCorp Sentinel and Open Policy Agent (OPA) are popular frameworks for PaC, working with infrastructure as code (IaC) tools like Terraform. Unlike traditional IaC, which configures infrastructure, PaC sets up policy rules that are enforced whenever infrastructure changes are proposed. This approach helps maintain a secure, compliant cloud environment by preventing risky configurations.Behind the scenes of the OpenTelemetry Governance CommitteeThe OpenTelemetry Governance Committee (GC) guides the OpenTelemetry project strategically, ensuring its growth as a vendor-neutral observability framework. While the Technical Committee (TC) focuses on technical aspects, the GC's role includes setting project goals, updating policies, and overseeing SIG (Special Interest Group) sponsorships, ensuring alignment with community needs. GC members also represent OpenTelemetry at events, mediate conflicts, and check in with SIG maintainers to address challenges and gather feedback.⚡TechWave: Cloud News & AnalysisEC2 Image Builder now supports building and testing macOS imagesAWS EC2 Image Builder now supports creating macOS images, enabling users to streamline their image management and automate the creation of "golden images" (customized bootable OS images) for macOS in addition to Windows and Linux. This is particularly helpful for developers using macOS tools like Xcode and Fastlane, which are essential in CI/CD pipelines. With Image Builder, users can create components for specific tools, define a recipe for a base macOS image, configure infrastructure (like EC2 Mac Dedicated Hosts), and set up pipelines that automatically test and validate each image.Upgraded Claude 3.5 Sonnet from Anthropic (available now), computer use (public beta), and Claude 3.5 Haiku (coming soon) in Amazon BedrockAnthropic's latest updates to the Claude 3.5 model family in Amazon Bedrock include an upgraded Claude 3.5 Sonnet, which enhances the model’s ability to handle complex software engineering tasks, knowledge-based Q&A, data extraction, and task automation at the same cost as previous versions. Additionally, a new "computer use" feature, available in public beta, allows Claude 3.5 Sonnet to interact with computer interfaces, like opening applications, typing, and clicking, opening up possibilities for AI-driven automation in software testing and administrative workflows. Lastly, the upcoming Claude 3.5 Haiku will offer faster response times paired with strong reasoning abilities, ideal for applications requiring both speed and intelligence, such as customer service and data processing in sectors like finance and healthcare.Grafana 11.3 release: Scenes-powered dashboards, visualization and panel updates, and moreGrafana 11.3 introduces a range of new features and improvements, with a highlight on the new Scenes-powered dashboards, enhancing stability, flexibility, and organization of dashboard elements. This release also includes visual and functional updates, like a redesigned inspect feature for table cells, enabling quick data analysis, and the new "Actions" option, allowing users to trigger API calls directly from elements on canvas panels. The update further enhances alerting with simplified rule creation and RBAC for notifications, and Explore Logs is now a default feature, making log troubleshooting more accessible.Sonar Details OpenAPI Generator Flaw That Creates Source Code VulnerabilitySonar recently identified a vulnerability in the OpenAPI Generator, a popular tool for creating API libraries, that could allow attackers to read or delete files in certain directories. Although a patch has been released, many existing APIs built with older, unpatched versions might still be at risk, requiring DevSecOps teams to locate and update them. This vulnerability underscores the challenge of detecting security flaws in auto-generated code, where developers may be less involved in the underlying code creation process. With cybercriminals actively searching for such vulnerabilities, DevSecOps teams must prioritize remediating high-risk code while balancing limited resources.HashiCorp Updates Terraform; Wider Cloud Infrastructure Developer ToolsetsHashiCorp, now under IBM's ownership, announced significant updates to Terraform at HashiConf, focusing on streamlining multi-cloud infrastructure management. Terraform's new "stacks" feature allows developers to manage complex, interdependent infrastructure configurations, making it easier to scale and control cloud resources across multiple environments. Additionally, HCP Waypoint provides a structured portal for internal development, using templates to standardize application deployment and updates. Other enhancements include new lifecycle management capabilities for HCP Vault, GPU resource sharing in Nomad, and an automation tool for migrating Terraform workflows, all designed to optimize and automate infrastructure in an increasingly complex cloud landscape.🛠️HackHub: Best Tools for Cloudkubectl-guard: Accidentally modifying production instead of a local cluster? kubectl-guard helps prevent such critical mistakes.To set up *kubectl-guard*, first create a file named *kubectl-guard* for the script, then make it executable by running `chmod +x kubectl-guard`. Next, open your shell configuration file (e.g., `~/.zshrc`) in a text editor, and add an alias with the command `alias kubectl='full-path-to/kubectl-guard'`, replacing "full-path-to" with the actual path where the script is saved. Save and close the file, then restart your terminal session for changes to take effect. This setup will help ensure safety by requiring the production cluster name to include "prod," though you can adjust this by modifying the `PROD_IDENTIFIER` variable.kubesafe: Safely manage multiple Kubernetes clusters by defining safe contexts and protected commands.*Kubesafe* is a tool designed to help you avoid running risky commands on the wrong Kubernetes cluster by marking certain contexts as "safe" and defining commands that need confirmation before execution. It works with any Kubernetes CLI tool (like `kubectl` or `helm`) by wrapping the command to add this layer of protection. For instance, running `kubesafe kubectl delete pod my-pod` will prompt for confirmation if the context is marked as protected. You can set up aliases, such as `alias kubectl='kubesafe kubectl'`, to automatically use Kubesafe each time you run a command.Tfreveal:An open-source tool that enhances Terraform plan visibility by showing all resource and output differences, including sensitive values.*tfreveal* is an open-source tool that lets you see all changes, including sensitive values, in Terraform plan files, enhancing transparency in infrastructure updates. While Terraform hides sensitive data by default, tfreveal unearths these details, which is particularly useful for detecting drift between Terraform state and actual infrastructure. Typically, sensitive data can only be viewed through complex JSON outputs, making it hard to read, especially when changes are in large encoded values. tfreveal simplifies this by displaying clear diffs, showing all values. To use, generate a plan file with `terraform plan -out plan.out`, then pipe it to tfreveal via `terraform show -json plan.out | tfreveal`.SyncLite:A low-code platform for relational data consolidation, ideal for building data-intensive apps across edge, desktop, and mobile environments.SyncLite is an open-source, low-code platform for creating data-intensive applications that seamlessly consolidate and synchronize data across edge, desktop, and mobile environments. It supports real-time, transactional data replication from various sources, like embedded databases (e.g., SQLite, DuckDB) and IoT message brokers, and integrates with popular data destinations, such as databases, data warehouses, and data lakes.pg_replicate`pg_replicate` is a Rust library designed to help developers quickly set up data replication from PostgreSQL to various data systems. It simplifies the use of PostgreSQL’s logical streaming replication protocol, letting users focus on building data pipelines without dealing with protocol details. To get started, users create a PostgreSQL publication, run the stdout example to replicate data to standard output, and connect using simple commands.📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

A clean way to mount read-only image data in KubernetesCloudPro #90: Running as root without being root: Kubernetes finally gets user namespaces rightLive Webinar | Scale AppSec with Security Champions – May 15Security Champions programs are a proven way to scale AppSec across dev teams. Join Snyk’s live webinar on May 15 @ 11AM ET where we’ll cover👇✓ Defining the role of security champions✓ Designing a scalable, tailored program✓ Recognizing, rewarding & growing your champions🎓 BONUS: Earn CPE credits for attending!Save your spot!🔐 Cloud SecurityWe once let a vendor upload files directly into our S3 bucket. It was fine, until it wasn’t.If you’re building GenAI on AWS and winging the security architecture, start here instead.How I use LLMs as a staff engineerOur infra used to depend on whoever remembered how they set it up last time. This fixed it. How stable is your cloud infrastructure security?[Sponsored] When attacking digital wallets and SoftPOS mobile apps, threat actors target more than just data. Learn how to protect your digital wallets and SoftPOS apps.⚙️Infrastructure & DevOpsWe killed long-lived EC2 access and didn’t lose a minute of on-call response time. We were bleeding CloudWatch costs and had no idea who was hitting GetMetricData. This finally gave us the receipts.Built an HTTP1 server inside HTTP2, inside HTTP3, all for funIf SQL ever felt backwards to read, BigQuery’s pipe syntax fixes that. We needed rotating IPs and secure access for outbound traffic, This Squid on Fargate setup nailed it📦Kubernetes & Cloud NativeRunning as root without being root: Kubernetes finally gets user namespaces rightA clean way to mount read-only image data in KubernetesWhat’s new in CNCF Sandbox? 14 fresh projects shaping cloud native in 2024Securing the Kubernetes host operating systemNever Trust the Pod: Protecting Your Kubernetes Host the Right Way[Sponsored]Learn practical mobile app security tips to help mitigate attacks on SoftPOS and digital wallets.🔍 Observability & SREIf your ETL pipeline breaks and you’re still guessing where- read thisDon’t wait for your nodes to fail at 2AM- EKS can fix them for you nowWe ditched Prometheus for VictoriaMetrics: cut costs by 30%, and our slowest queries dropped from 30s to 3Making observability work: How platform engineers can improve monitoringSmall teams need PaaS-Ops, not DevOpsCheers,Shreyans SinghEditor-in-ChiefNew developer products provide a glimpse into the future of app building on HubSpot, including deeper extensibility, flexible UI, modern development tools, and moreHubSpot’s AI-powered ecosystem presents a global opportunity projected to reach $10.2 billion by 2028.To capitalize on that growth potential, we are opening our platform more, starting with expanded APIs, customizable app UI, and tools that better support a unified data stra`tegy.Start Building TodayForward to a Friend📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Applying SRE principles to your MLOps pipelinesCloudPro #81: Securing DeepSeek and other AI systems with Microsoft Security⭐MasterclassAmazon S3 now supports appending data to an objectThe dangers of Terraform automation platformsHyrum's law in GolangCloud dev environmentsA Deep Look into Our New Massive Multitenant ArchitectureTesting with Go and PostgreSQL: Learn how to use ephemeral PostgreSQL instances in Go to simplify tests, reduce resource usage, and improve CI efficiency. Practical tips included.Creating an AMI with Image Builder:Learn to use Packer and Terraform to create pipelines, manage custom AMIs, and streamline automation.Designing a Zero Downtime Migration:Complete guide on Docker: Learn to build and deploy your distributed applications easily to the cloud with Docker.SQL Style Guide: This SQL Style Guide outlines best practices for writing clear and maintainable SQL code.🔍Secret KnowledgeSemantic Versioning to Simplify Release Management:Learn how to use Semantic Versioning with semantic-release and GitHub Actions in your AWS CDK project.How to Use Blocks in Ansible Playbooks:How to use Ansible blocks to group tasks, handle errors gracefully, and manage cleanup operations effectively in your playbooks.Monitoring PM2 in production: Learn how to monitor PM2-managed Node.js apps in New Relic using Flex, capturing key metrics like CPU, memory, and logs with a streamlined setup.Infra as Code with CDK for Terraform: Learn how Zip’s security team used Python CDK for Terraform to enforce security guardrails on AWS infrastructure.Ingesting CloudWatch Logs into OpenSearch: Sample code to showcase ingestion of Amazon CloudWatch logs into Amazon OpenSearch Serverless.AWS Cost for Home Assistant: This tutorial showcases how to get the month-to-date cost and forecast cost and make it available on Home Assistant.Installing Windows XP in DOSBox-X: Learn how to install Windows XP on DOSBox-X, even though it’s not officially supported.Solve Missing AWS Resources in Terraform: The awscc provider, using AWS's Cloud Control API, helps overcome resource gaps.Optimizing PostgreSQL Queries at Scale: Tips to troubleshoot inefficient queries and resource bottlenecks.Getting Started with Terraform Stack: Simplify deployments across environments like dev, staging, and production without complex CI/CD setups.⚡TechwaveIntroducing the AWS Trust CenterAWS CloudTrail network activity events for VPC endpoints now generally availableAWS Verified Access support for non-HTTP resources is now generally availableSecuring DeepSeek and other AI systems with Microsoft SecurityApplying SRE principles to your MLOps pipelinesGrafana Cloud updates: Exemptions in Adaptive Logs, GPU monitoring in AI Observability, and moreIntroducing RHEL AI 1.4: Powering the Next Wave of Generative AI InnovationAlibaba, Datadog, and Quesma Join Forces on Go Compile-Time InstrumentationEnrich Your On-Call Experience With Observability Data at Your Fingertips by Using Datadog On-CallCheers,Shreyans SinghEditor-in-ChiefAWS Cloud ProjectsLately, I've been on the lookout for books that don’t just explain AWS concepts but actually help you build something useful. AWS Cloud Projects caught my attention because it takes a hands-on approach—walking through projects that range from hosting a website to building AI-powered applications. If you're like me and prefer learning by doing, check this out.GET ITForward to a Friend📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Policy as code in Kubernetes: security with seccomp and network policiesCloudPro #80: Policy as code in Kubernetes: security with seccomp and network policiesWe want to makeCloudProeven better for professionals like you! Take ourquick 10-minute surveyand help shape the content we create. As a thank you, you'll receive a freePackteBook (worth $20) and have the option to participate in a paid user interview.Take the Survey NowThe Ultimate Linux Shell Scripting Guide: Automate, Optimize, and Empower tasks with Linux Shell ScriptingI’ve always believed that knowing how to write solid shell scripts is an underrated superpower. Whether you're automating tasks, troubleshooting servers, or just making your daily workflow smoother, scripting saves time and headaches. The Ultimate Linux Shell Scripting Guide caught my eye because it doesn’t just focus on Bash—it also dives into Zsh and even PowerShell on Linux. If you’re looking to sharpen your command-line skills and build real-world automation, this one is worth a read.GET IT⭐MasterclassPolicy as code in Kubernetes: security with seccomp and network policiesThe Kubernetes introduction I wish I had when I started: A complete guideA practitioner’s guide to wide eventsLonghorn backup and restoreKubernetes CI/CD pipelines with CircleCI and DevtronMoving off Heroku, slowlyDo you really need Redis?Discovering hidden vulnerabilities in Portainer with CodeQLWhat’s new with Robinhood, our in-house load-balancing serviceWhat is Helm in Kubernetes? A complete guide🔍Secret KnowledgeReducing the cost of a Google Cloud Dataflow Pipeline by over 60%Reflections on IaC using TerraformHow to Improve Your DevOps AutomationToo Many MicroservicesEnhance Argo CD observability: A step-by-step guide to integrating Prometheus metrics into the UIParsing all the data with open source tools: Unstructured and PgaiMonitoring MySQL using Prometheus Exporter and GrafanaThat time when KinD stopped working in GitHub CodespacesIngesting F1 Telemetry UDP real-time data in AWS EKS⚡TechwaveAWS CDK is splitting Construct Library and CLINew Amazon S3 Tables: Storage optimized for analytics workloadsSpanner Graph is now GAAnnouncing Gen AI Toolbox for Databases by Google CloudGrafana 11.5 release: easily share Grafana dashboards and panels, secure frontend code for plugins, and moreDigma Adds Ability to Predict Coding Issues to Observability PlatformThe Cloud Controller Manager Chicken and Egg ProblemAmazon EC2 now supports automated recovery of Microsoft SQL Server with VSSGKE’s faster cluster upgrades under the hoodAWS CloudTrail network activity events for VPC endpoints now generally availableCheers,Shreyans SinghEditor-in-ChiefAWS Cloud ProjectsLately, I've been on the lookout for books that don’t just explain AWS concepts but actually help you build something useful. AWS Cloud Projects caught my attention because it takes a hands-on approach—walking through projects that range from hosting a website to building AI-powered applications. If you're like me and prefer learning by doing, check this out.GET ITForward to a Friend📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Automating Windows Server Management with PowerShellWe want to make CloudPro even better for professionals like you! Take our quick 10-minute survey and help shape the content we create. As a thank you, you'll receive a free Packt eBook (worth $20) and have the option to participate in a paid user interview.Take the Survey NowCloudPro: Special IssueHey there,As cloud professionals, we are always looking for ways to improve our skills and build solutions that are scalable, secure, and efficient. While regular news and updates keep us informed, sometimes it's good to take a deep dive into topics that matter.That’s why we’re bringing you this special issue of CloudPro, where we explore two carefully selected books that provide practical, hands-on learning experiences.The first book, Platform Engineering for Architects, takes a deep dive into building and maintaining internal developer platforms. We’ll explore a hands-on section on Building a Self-Service Internal Developer Platform with Terraform, helping you understand Infrastructure as Code (IaC) in action.The second book, Windows Server 2025 Administration Fundamentals, goes beyond basic administration to teach PowerShell automation techniques for managing Windows Server environments. We've included a technical excerpt on Automating Active Directory User Management, a crucial skill for IT administrators.The third book, Cloud Observability with Azure Monitor, provides in-depth guidance on monitoring cloud infrastructure. We’ve included a highly practical excerpt on Configuring Azure Monitor for Real-Time Log Analysis, showing how to use KQL queries to detect performance issues before they impact users.If you’re serious about learning by doing, this issue is for you. Dive in and explore!Platform Engineering for ArchitectsBuilding a Self-Service Internal Developer Platform (IDP)One of the primary objectives of platform engineering is to provide an internal developer platform (IDP) that automates infrastructure provisioning. Below is a Terraform snippet to provision a Kubernetes cluster using Infrastructure as Code (IaC):provider "aws" { region = "us-east-1"}resource "aws_eks_cluster" "example" { name = "example-cluster" role_arn = aws_iam_role.example.arn vpc_config { subnet_ids = [aws_subnet.example1.id, aws_subnet.example2.id] }}After applying this configuration, developers can interact with the cluster using kubectl with minimal setup.Why This Matters:Automating platform setup reduces cognitive load on developers, improves consistency, and ensures security best practices are followed.GET IT HEREWindows Server 2025 Administration FundamentalsAutomating Windows Server Management with PowerShellPowerShell scripting is crucial for automating administrative tasks in Windows Server. Below is a script to create, configure, and secure a new Active Directory user automatically:# Define user parameters$UserName = "jdoe"$Password = ConvertTo-SecureString "SecureP@ssw0rd" -AsPlainText -Force# Create the user in Active DirectoryNew-ADUser -Name "John Doe" -SamAccountName $UserName -UserPrincipalName "[email protected]" -Path "OU=Users,DC=example,DC=com" -AccountPassword $Password -Enabled $true# Add user to a security groupAdd-ADGroupMember -Identity "Developers" -Members $UserNameWhy This Matters:Manually managing user accounts is inefficient and error-prone. Automating it ensures compliance and operational efficiency.GET IT HERECloud Observability with Azure MonitorConfiguring Azure Monitor for Real-Time Log AnalysisAzure Monitor helps track and analyze cloud infrastructure performance. The following Kusto Query Language (KQL) query identifies virtual machines experiencing high CPU usage:Perf| where ObjectName == "Processor" and CounterName == "% Processor Time"| summarize AvgCPU = avg(CounterValue) by Computer, bin(TimeGenerated, 5m)| where AvgCPU > 80| order by AvgCPU descTo run this query in Azure Log Analytics, follow these steps:=> Navigate to Azure Monitor > Logs.=> Select your Log Analytics Workspace.=> Paste the KQL query and click Run.Why This Matters: Proactive monitoring ensures infrastructure stability. Automating performance alerts reduces downtime and enhances reliability.GET IT HERE📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Announcing CDK Garbage CollectionCloudPro #82: NFTables mode for kube-proxy | KubernetesThe Self-Taught Cloud Computing EngineerBreaking into cloud computing can feel overwhelming, especially when juggling AWS, Azure, and GCP. The Self-Taught Cloud Computing Engineer stands out because it provides a structured, hands-on approach to mastering all three major cloud platforms.Whether you're aiming for certifications, career growth, or just a deeper understanding, this book walks you through real-world projects and practical skills that matter. If you're serious about leveling up in cloud computing, this is one to check out.GET IT⭐MasterclassFinding Vulnerabilities at Scale: How a JPEG processing vulnerability led to discovering security flaws in major projects like Chromium and WINE.Simplifying Multi-Environment Kubernetes Deployments: Kluctl automates deployments, integrates with Helm and Kustomize, and reduces manual intervention.How to Structure a Terraform Project: The tutorial covers approaches like monorepo and polyrepo setups for managing environments and modules.What is Inference Parallelism and how it worksGuide to deploy SpinKube with WASM on Taikun CloudWorksGo faster! Optimizing Golang for performance and scaleFlatcar brings Container Linux to the CNCF Incubator347 Million reasons to manage HashiCorp Vault as codeSpotlight on Kubernetes upstream training in JapanOpenSSF Expands secure development course with Interactive Labs🔍Secret KnowledgeDeploy Azure Resources from GitLab with No Secrets Using OpenTofu: Learn how to deploy Azure resources using GitLab pipelines with OpenID Connect, all without managing secrets.Promoting Terraform Changes from DEV to PROD: Learn how to use workspaces for separate states and environment folders to clearly separate multi-envs deployments.Implementing Compliant Secrets with AWS Secrets Manager:This tutorial walks you through building detailed access policies and introduces a Terraform module to automate and simplify policy management.Terraform Stacks with Azure:This guide covers dynamic credentials, creating stacks with multiple components, and using orchestration rules for automatic deployment approvals.Why Falco’s new response engine is a game changer for open source cloud native securityThere and back again: Port forwarding with mirrordKCD UK slides: Brownfield realities, platforms orchestration & app devsHow to enter Kubestronaut orbit and beyondA beginner’s guide to progressive delivery of a cloud native applicationFlatcar accepted into CNCF at incubating level⚡TechwaveNFTables mode for kube-proxy | KubernetesAnnouncing CDK Garbage CollectionAnnouncing the general availability of AWS .NET OpenTelemetry librariesAnnouncing new models, customization tools, and enterprise agent upgrades in Azure AI FoundryEmpowering innovation: The next generation of the Phi familyNew Terraform provider for Oracle Database@Google CloudNew Cloud Trace features to troubleshoot latency and errorsAutomating IT Network support with watsonx and Juniper’s Mist AIAmazon ECS increases the CPU limit for ECS tasks to 192 vCPUsAWS Network Firewall introduces automated domain lists and insights🛠️Hackhub:Booster Framework: create event-driven backend microservices that focus on extreme development productivityyunionio/cloudpods: A cloud-native open-source unified multi-cloud and hybrid-cloud platformvmware-tanzu/velero: Backup and migrate Kubernetes applications and their persistent volumestsypuk/aws-client-monitor: advanced monitoring of aws client (both aws cli and any language aws sdk)sjramblings/ebsight: EBSight: Intelligent EBS Volume Analyzerturbot/tailpipe: select * from logs; Tailpipe is an open source SIEM for instant log insights, powered by DuckDB. Analyze millions of events in seconds, right from your terminal.aws-samples/aws-health-events-insight: centralized approach to store and analyze AWS Health events(PHD,SHD)dannysteenman/vscode-iam-service-principal-snippets: VS Code extension that provides autocompletion of all AWS services that can be used as Service Principals in your IAM policies.aws-samples/bedrock-engineer: Autonomous software development agent apps using Amazon Bedrock, capable of customize to create/edit files, execute commands, search the web, use knowledge base, use multi-agents, generative images and more.awslabs/StsSamlDriverCheers,Shreyans SinghEditor-in-ChiefForward to a Friend📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

The Lost Fourth Pillar of ObservabilityCloudPro #92Sponsored: Most GenAI projects die in the proof-of-concept stage. This session by Rubrik shows you how to push past that👇Save Your SpotThis week’s CloudPro issue has got a bunch of things I’ve either run into myself or seen others get tripped up by:📌AWS defaults that quietly expose more than they should📌a Kubernetes bug that’s been around for ten years📌GitHub Actions setups that look fine until someone finds a way inThere’s also a couple posts you'll find helpful, like building a CI/CD pipeline that’s actually fast, or understanding how containers really run under the hood.Hope a few of these come in handy when you need them.Also: we’re planning another special issue for next week. Any ideas on what we should dive into, or an expert you’d love to hear from? Just reply back to this email, I’d really like to hear what you think.Cheers,Shreyans SinghEditor-in-Chief🔐 Cloud SecurityAmazon GuardDuty Malware Protection for EC2 now available in AWS GovCloud (US) RegionsAmazon has released malware protection for EC2 in AWS GovCloud (US) regions. It scans EBS volumes attached to EC2 instances and container workloads to detect potential malware. The system supports both automatic scans based on suspicious behavior and manual scans using the EC2 instance's ARN. It works without adding any new software and does not impact workload performance.Amazon VPC adds CloudTrail logging for VPC resources created by defaultAmazon VPC now logs creation and deletion of default resources—like Security Groups, Route Tables, and Network ACLs, when a VPC is created or deleted. Previously, CloudTrail only captured explicitly created resources, making audits harder. This update helps teams improve governance and track changes more easily.Guardrails for Your Cloud: A Simple Guide to OPA and TerraformThis post shows how to use OPA to block risky Terraform changes like unencrypted S3 buckets or open security groups. It explains how to write Rego policies, run checks on Terraform plans, and enforce standards like required tags and deployment restrictions. Helpful for adding policy-as-code guardrails to IaC workflows.Shadow Roles: AWS Defaults Can Lead to Service TakeoverThis research shows how default AWS service roles, like those for SageMaker, Glue, and EMR, often come with overly broad S3 permissions, such as AmazonS3FullAccess. Attackers can abuse these defaults to escalate privileges and compromise other services. Real-world scenarios include model-based attacks via Hugging Face and cross-service takeovers through default IAM roles.Hardening GitHub Actions: Lessons from Recent AttacksTwo recent supply chain attacks exploited weak GitHub Actions workflows, compromising popular repos via over-permissive settings and exposed secrets. The report urges tighter defaults: set tokens to read-only, limit third-party Actions, avoid risky triggers like pull_request_target, and never expose secrets to forks. It also warns self-hosted runners can be dangerous if shared or persistent.Build Your Own AI Agents Over The WeekendJoin the live "Building AI Agents Over the Weekend" Workshop starting on June 21st and build your own agent in 2 weekend. In this workshop, the Instructors will guide you through building a fully functional autonomous agent and show you exactly how to deploy it in the real world.BOOK NOW AND SAVE 35%Use Code AGENT35 at checkout⚙️ Infrastructure & DevOpsRedis Is Open Source AgainRedis has shifted back to an open source license (AGPLv3) for Redis 8 after a year under more restrictive licenses meant to block cloud providers from monetizing it freely. The pivot follows the rise of the Valkey fork, backed by AWS and Google, and a recognition that Redis had lost favor with parts of the developer community.37signals Says Goodbye to AWS: Full S3 Migration and $10M in Projected Savings37signals has fully migrated 18 PB of data off AWS S3 to its own Pure Storage-based infrastructure, ending over a decade on the platform. AWS waived the $250K egress fee, aligning with EU Data Act requirements. The company expects to cut infrastructure costs from $3.2M to under $1M annually, saving over $10M in five years.Docker Explained: Finally Understand Containers Without Losing Your Mind (Probably)This post explains how Docker packages your code and dependencies into isolated containers that run the same everywhere. It covers Dockerfiles, images, layers, and containers with clear examples. Useful for devs struggling with environment issues during deployment.How I Tuned My CI/CD Pipeline To Be Done in 60 SecondsA solo developer reduced their GitHub Actions CI/CD pipeline from over 5 minutes to under 60 seconds using parallel jobs, caching, and Makefile tuning. They optimized builds, tests, and linting while managing GitHub's billable minutes. The result: fast, repeatable deploys with zero YAML debugging overhead.Ultimate DevOps Roadmap 2025: Learn Automation, ContainerizationThis guide lays out a step-by-step DevOps learning plan for 2025, covering scripting, cloud, CI/CD, Kubernetes, IaC, and AIOps. It includes timelines, open-source tools, and free resources for each topic. Useful for engineers building a modern, automation-driven skillset from scratch.📦 Kubernetes & Cloud NativeKubernetes v1.33 Fixes a 10-Year-Old Image Pull LoopholeKubernetes v1.33 closes a decade-old loophole that let pods reuse cached private images without valid pull credentials. With a new Kubelet flag, image access is now authorized even if the image already exists on the node. This improves security in multi-tenant clusters using private registries.Announcing etcd v3.6.0The first etcd minor release in four years adds full downgrade support, better memory efficiency, and removes the deprecated v2store. It introduces Kubernetes-style feature gates, livez/readyz probes, and SIG-etcd governance under Kubernetes. A 50% memory drop and ~10% throughput boost make it the most optimized and robust release to date.Kubernetes API Groups Explained Like You’re 5: Why They Matter (With Real Examples)This post simplifies Kubernetes API groups using familiar YAML examples like apps/v1 and rbac(.)authorization(.)k8s(.)io/v1. It breaks down how resources are grouped and versioned to help engineers better navigate manifests. A useful primer for anyone confused by Kubernetes API structure.Kubernetes Production ChecklistThis post offers a detailed checklist of proven Kubernetes production best practices—from health checks and autoscaling to RBAC, secrets, and observability. It covers what really matters for keeping systems secure, resilient, and scalable in real-world environments.Building Kubernetes (a lite version) from scratch in GoThis project walks through building a simplified Kubernetes clone in Go, recreating the control plane, scheduler, and kubelet logic using HTTP APIs and in-memory storage. It’s a hands-on way to demystify how reconciliation loops and pod lifecycles work under the hood.🔍 Observability & SREIntroducing the OTTL Playground for OpenTelemetryElastic has launched OTTL Playground, a browser-based tool for testing OpenTelemetry Transformation Language (OTTL) statements in real time. It lets users run processors like transform and filter, view diffs, logs, and JSON outputs, and safely test transformations without affecting production. It’s built with WebAssembly and offers shareable config links for easier collaboration.Last9 MCP Server: Fix Production Issues in Your Local EnvironmentLast9 has launched MCP Server, a tool that brings real production exceptions (with full context) into your local dev environment. It captures stack traces, request parameters, and environment variables so bugs can be reproduced and fixed precisely where you're coding. It integrates with AI agents in editors like Claude (via Cursor, Windsurf) to auto-suggest fixes, cutting debug time by over 35%.The Lost Fourth Pillar of ObservabilityCloudQuery argues that configuration data, unlike logs, metrics, and traces, offers crucial insights without needing instrumentation. It’s high-cardinality, API-collected, and best stored relationally. Monitoring config data helps track security posture, compliance, cost leaks, and infrastructure drift. Integrating it with traditional observability sharpens root cause analysis and preemptive alerting.A tcpdump Tutorial with ExamplesDaniel Miessler’s tutorial breaks down tcpdump into 50 real-world examples for capturing and analyzing network traffic. From filtering by IP, port, and protocol to saving captures and flag-specific filters, it’s a compact field guide for security engineers and SREs. Great for fast, precise troubleshooting from the command line.How Kubernetes Runs Containers : A Practical Deep DiveThis tutorial breaks down how Kubernetes runs containers by tracing a pod’s lifecycle on a Linux VM using k3s, crictl, and pstree. It shows how pods are just Linux processes isolated by namespaces and cgroups, with container runtimes like containerd managing their lifecycle. This clarity helps engineers debug resource limits, network issues, and process isolation at a low level.Forward to a Friend📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Troy Hunt: Processing 23 Billion Rows of ALIEN TXTBASE Stealer LogsCloudPro #82: Kubernetes History Inspector (KHI) visualizes cluster logs: by Google CloudYour Salesforce Data, Your Responsibility: Best Practices for Data ProtectionLearn More⭐MasterclassStateful apps in Kubernetes. From history and fundamentals to operatorsStateful applications in Kubernetes require additional attention due to their need to persist data across instances and reboots, unlike stateless applications.Backstage on KubernetesIn this article, you'll learn how to integrate Backstage with Kubernetes, first by running Backstage outside the cluster using the Kubernetes API, and then deploying it directly on the cluster with the official Helm chart.Manage secrets in AWS EKS with AWS Secrets Manager securelyAWS Secrets Manager integrates with EKS through the AWS Secrets and Configuration Provider (ASCP) for the Kubernetes Secrets Store CSI Driver.Your guide to observability engineering in 2024In 2024, an observability engineer's role is multifaceted, requiring expertise in data pipelines, system analysis, and troubleshooting to maintain and optimize complex, distributed systems.The complete guide to serverless appsWhile the term implies the absence of servers, it actually refers to a model where developers don't need to manage server infrastructure. Instead, the cloud provider handles server management🔍Secret KnowledgeHow Stripe’s Document Databases Supported 99.999% Uptime with Zero-Downtime Data MigrationsThis reliability is largely due to their custom-built database infrastructure, called DocDB, which is an extension of MongoDB Community.Anomaly Alerting in PrometheusUsing Prometheus with Istio, we can set up a generic anomaly detection system for response times that applies to all services running on a mesh.The ROI of improving and investing in DORADORA Metrics provide a comprehensive view of your software team's performance, helping identify areas for improvement and measure the return on investment (ROI) of these enhancements.AWS Managed KMS Keys and their Key Policies: Security Implications and Coverage for AWS ServicesAWS Managed KMS Keys are encryption keys managed by AWS but used within your own AWS account, often applied as default keys for various services.Attack Paths Into VMs in the CloudThis post reviews how attackers might exploit VMs and offers strategies for organizations to secure their environments.⚡TechwaveKubernetes History Inspector (KHI) visualizes cluster logs: by Google CloudTroy Hunt: Processing 23 Billion Rows of ALIEN TXTBASE Stealer LogsGet insights from multimodal content with Amazon Bedrock Data Automation, now generally availableOptimizing incident management with AIOps using the Triangle SystemNew Cloud Trace features to troubleshoot latency and errors🛠️HackhubSQLSync: collaborative offline-first wrapper around SQLite. It is designed to synchronize web application state between users, devices, and the edge.lambda_helpers_metrics:A library that simplifies sending custom metrics to CloudWatch using EMF (Embedded Metric Format).cloudysetup: Automate cloud environment setup using generative AI and AWS Cloud Control API.kftray: A cross-platform system tray application for managing multiple kubectl port-forward commands, with support for UDP and proxy connections through k8s clusterskeycloak: Open Source Identity and Access Management For Modern Applications and ServicesCheers,Shreyans SinghEditor-in-ChiefProtect Data Privacy and Optimize AI Models with Tonic TextualLLMs have tapped all of publicly available data. The last mile training of models requires private data. Use private data without compromising security. Redact, label, and prep freetext for LLM ingestion or data pipelines.Start Free TrialForward to a Friend📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

How Postgres stores data on diskCloudPro #75: A Guide to Kubernetes Network Policies⭐MasterclassA Guide to Kubernetes Network PoliciesDockerfile Instructions - ADD vs. COPYHow to add new worker node to existing Kubernetes clusterHow I Reduced Docker Image Size from 588 MB to Only 47.7 MBAmbient mesh: Can sidecar-less Istio make your application faster?🔍Secret KnowledgeOops, I Deleted the AWS Auth RolesRising Incidents on Git PlatformsHow Postgres stores data on diskHow We Integrate a New Service in Under 1 Hour for 25 ClustersEleventeen ways to delete an AWS resource🛠️HackhubApeman: AWS attack path management toolCyphernetes: A Kubernetes Query LanguageDesed:A command-line tool for complex sed scriptsKueue: Kubernetes-native Job QueueingAWS CloudFormation StarterkitCheers,Shreyans SinghEditor-in-Chief⭐MasterClass: Tutorials & GuidesA Guide to Kubernetes Network PoliciesIn Kubernetes, network policies control the traffic between pods, ensuring secure communication within the cluster. There are two main types: Layer 4 (L4) and Layer 7 (L7) policies. L4 policies manage traffic at the transport layer (e.g., TCP/UDP) based on IP addresses and ports, while L7 policies operate at the application layer (e.g., HTTP) with more fine-grained control over communication between services. L7 policies often require a service mesh like Linkerd, which adds features like mutual TLS (mTLS) for encrypted communication.Dockerfile Instructions - ADD vs. COPY`COPY` is simple and secure, only transferring files from the local build context to the image. In contrast, `ADD` offers extra functionality, such as downloading files from URLs or automatically extracting compressed archives. However, this added flexibility introduces complexity and potential security risks. Best practice recommends using `COPY` for most cases due to its straightforwardness, reserving `ADD` for situations where its unique features are necessary.How to add new worker node to existing Kubernetes clusterTo add a new worker node to an existing Kubernetes cluster, start by setting up a new Ubuntu 24.04 instance and configuring its hostname and `/etc/hosts` file. Disable swap memory, load necessary kernel modules, and install containerd as the container runtime. Add the Kubernetes APT repository, then install Kubernetes components like kubeadm, kubelet, and kubectl. On the control plane node, generate a kubeadm join command with a token. Run this command on the new worker node to join the cluster. Finally, verify the addition by checking the nodes from the control plane using `kubectl get nodes`.How I Reduced Docker Image Size from 588 MB to Only 47.7 MBTo significantly reduce a Docker image size, using multi-stage builds is key. In this case, a Flask app's image size was reduced from 588 MB to just 47.7 MB by switching to the lightweight Python 3.9-alpine image and using a multi-stage build approach. Multi-stage builds allow you to separate the build and runtime environments, keeping only essential runtime dependencies in the final image. Additionally, minimizing the number of layers by combining commands, using a `.dockerignore` file to exclude unnecessary files, and optimizing the Dockerfile structure contributed to this impressive 91.89% reduction.Ambient mesh: Can sidecar-less Istio make your application faster?Ambient mode in Istio, introduced in 2022, allows a sidecar-less architecture that can sometimes make applications faster. In traditional service meshes, adding latency is expected, but tests with ambient mode showed slightly improved performance in some cases, like the Bookinfo application's details service. This is partly because of more efficient connection handling and reduced syscalls in ambient mode, which offsets the overhead of extra hops via lightweight ztunnels.🔍Secret Knowledge: Learning ResourcesOops, I Deleted the AWS Auth RolesThe author, while managing an EKS (Elastic Kubernetes Service) cluster using Terraform, accidentally deleted the AWS authentication roles, which are crucial for accessing the cluster. This resulted in losing access to the EKS cluster. The fix involved manually restoring access by modifying the EKS API access configuration via the AWS Console, re-adding the necessary admin roles, and regenerating the `aws-auth` config map.Rising Incidents on Git PlatformsIn 2023, incidents affecting popular DevOps platforms like GitHub, Bitbucket, GitLab, and Jira increased, with issues such as RepoJacking, security vulnerabilities, and performance disruptions. GitHub saw a rise in attacks, with hackers exploiting vulnerabilities and hosting malware. Atlassian products like Bitbucket and Jira faced security flaws, with Jira experiencing a significant increase in incidents. GitLab suffered from performance issues and security breaches, including a major Proxyjacking attack.How Postgres stores data on diskPostgres stores data on disk in a well-organized, file-based structure within a directory, typically located at `/var/lib/postgresql/data`. Inside this directory, you'll find folders like `base/`, where actual database data for each database is stored, and `pg_wal/`, which holds the Write-Ahead Log (WAL) files that help recover data after crashes. Each table and database object is ultimately represented by files in these directories. PostgreSQL uses clever abstractions to manage data, such as snapshots for transactions, dynamic shared memory for handling multiple processes, and special mechanisms like tablespaces for physically separating certain data.How We Integrate a New Service in Under 1 Hour for 25 ClustersThe article describes how a team integrated a new service called Otterize across 25 clusters in under an hour, emphasizing that while the technical setup was quick, the lengthy licensing process took over four months. The integration involved automating several steps using GitOps and tools like Argo CD to avoid manual errors. Key tasks included creating an organization and environment, inviting users, integrating with Kubernetes, securely managing credentials, and deploying the setup through a script.Eleventeen ways to delete an AWS resourceOur goal is to reduce AWS costs, but the deletion methods vary widely, often leaving users frustrated. They categorize deletion patterns, from simple one-click deletes to more complex confirmations that require typing specific phrases or acknowledging consequences. Ultimately, AWS should standardize its deletion processes to improve user experience and security, and they call for more data on user behavior during these actions.🛠️HackHub: Best Tools for CloudApeman: AWS attack path management toolProject Apeman is an AWS attack path management tool that helps analyze and manage AWS security data. To set it up, you need Docker, Python, and a virtual environment. Once the system is initialized, Apeman gathers AWS account data, including authorization details and ARNs, which are then ingested into a graph database for analysis.Cyphernetes: A Kubernetes Query LanguageCyphernetes is a Cypher-inspired query language for Kubernetes, simplifying complex Kubernetes operations with intuitive, SQL-like queries. It allows developers to easily manage Kubernetes resources by expressing relationships between them, such as connecting deployments to services and ingresses.Desed:A command-line tool for complex sed scriptsDesed is a command-line tool designed to help debug and understand complex `sed` scripts. It allows users to step through their scripts, both forwards and backwards, preview how substitute commands will affect the pattern space, and set breakpoints to examine the program's state. Desed also supports hot reloading, so changes to the source code can be instantly applied without restarting the debugger.Kueue: Kubernetes-native Job QueueingKueue is a Kubernetes-native job queueing system that manages when jobs start and stop based on a variety of factors, such as priorities and resource availability. It offers features like job management with FIFO strategies, resource fair sharing, dynamic resource reclaim, and integration with popular job types like BatchJob and Kubeflow training jobs.AWS CloudFormation StarterkitAn AWS CloudFormation starterkit including CI/CD and dev tools that allow you to securely and quickly deploy CloudFormation stacks on your AWS account.📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Google Kubernetes Engine supports 65,000-node clustersCloudPro #76: Kubernetes Gateway API v1.2: WebSockets, Timeouts, Retries, and MoreMastering Software Deployments at the Edge: A User’s Guide to Diverting DisasterSoftware delivery to dedicated edge devices is one of the most complex challenges faced by IT professionals today. While edge deployments come with inherent complications, it’s possible to avoid the pitfalls. With this guide in hand, a little planning, and the right tools and strategies in place, you can be confident you’ll never push a faulty update at scale.Read the Guide⭐MasterclassI have asked this SSH question in every AWS interviewHow to Ace (CKAD) Certified Kubernetes Application Developer examKubernetes resource model, controller pattern and operator SDK refresherHow we avoided an outage caused by running out of IPs in EKSDeploying a Serverless REST API🔍Secret KnowledgeHow to Differentiate Manual Changes from Terraform Changes in S3 BucketManaging AWS EKS access entries with Terraform and OpenTofuUber’s billion trips migration setup30 Days of CNCF projects | Day 5: What is Crossplane + demoCI/CD automation with Tekton: GitHub PR pipeline guide⚡TechwaveKubernetes Gateway API v1.2: WebSockets, Timeouts, Retries, and MoreGoogle Kubernetes Engine supports 65,000-node clustersContainer Insights with enhanced observability now available in Amazon ECSNew Amazon S3 Tables: Storage optimized for analytics workloadsGrafana 11.4 release: Introducing support for OpenSearch PPL and OpenSearch SQL in the AWS CloudWatch data source plugin🛠️HackhubNovascans your cluster for installed Helm charts and then cross-checks them against all known Helm repositories.Pglite-fusionallows you to embed an SQLite database in your PostgreSQL table. AKA multitenancy has been solved.Drasiis a data processing platform that simplifies detecting changes in data and taking immediate action.SonarIACis a static code analyzer for Infrastructure-as-Code languages such as CloudFormation and Terraform, as well as DevOps like Docker and Kubernetes.Pg_flois a CLI to move and transform data between PostgreSQL databases using Logical Replication.Cheers,Shreyans SinghEditor-in-ChiefLearn Million Dollar AI Strategies & Tools in this 3 hour AI Training for Free.This 3 hour power packed workshop that will teach you 30+ AI Tools, make you a master of prompting & talk about hacks, strategies & secrets that only the top 1% know of.By the way, here’s sneak peek into what’s inside the training:-Making money using AI-The latest AI developments, like GPT o1-Creating an AI clone of yourself, that functions exactly like YOU-10 BRAND new AI tools to automate your work & cut work time by 50%1.5 Million people are already RAVING about this hands-on Training on AI Tools. Don’t take our word for it? Attend for yourself and see.Register here⭐MasterClass: Tutorials & GuidesI have asked this SSH question in every AWS interviewIn AWS interviews, a popular and insightful question is: "You're trying to SSH into an EC2 instance, but it’s failing. How would you troubleshoot?" While it seems simple, the question evaluates a candidate's problem-solving approach, understanding of AWS infrastructure, and real-world experience. A key expectation is that candidates check security groups first since they act as firewalls controlling traffic. Surprisingly, many overlook this basic yet crucial step, diving into more complex areas instead. The question thus highlights how well candidates understand AWS fundamentals and prioritize troubleshooting steps effectively.How to Ace (CKAD) Certified Kubernetes Application Developer examThe Certified Kubernetes Application Developer (CKAD) exam is a practical certification focused on Kubernetes application deployment, maintenance, and troubleshooting. Ideal for engineers managing containerized applications in Kubernetes, it tests real-world problem-solving skills across topics like application design, deployment strategies, observability, security, and networking. The exam includes hands-on tasks performed in a live Kubernetes cluster and allows access to documentation during the test. It's considered pre-professional in difficulty, with a 66% passing score and retake opportunities. Preparation involves mastering Kubernetes CLI commands, understanding concepts like pods, deployments, and ConfigMaps, and practicing with tools like Killer.sh to simulate the exam experience.Kubernetes resource model, controller pattern and operator SDK refresher] The Resource Model uses etcd as the state store, with resources defined by objects like Kind, Group, Version, and Resource, which are mapped to API endpoints (e.g., /apis/apps/v1/deployments). Informers and SharedInformers optimize resource management by efficiently watching changes in objects, reducing API server load. Informers utilize Reflectors to fetch and cache data, Listers to retrieve objects from the cache, and Workqueues to process events like Add, Update, or Delete. Controllers act as loops that continuously reconcile the current state of resources (from their status) with the desired state (defined in their spec).How we avoided an outage caused by running out of IPs in EKSAdevinta's platform team tackled the critical issue of IP exhaustion in their EKS clusters by implementing custom networking with a secondary CIDR to allocate additional IPs, avoiding potential outages. The problem stemmed from the VPC-CNI plugin's default behavior of assigning an IP address per pod, which strained available IPs in their VPC as clusters scaled. While alternatives like switching to Cilium or enabling IPv6 were explored, the chosen solution balanced speed and reliability, enabling the team to complete their migration to EKS. By carefully testing and rolling out custom networking, the team stabilized IP usage, avoided service disruptions, and ensured seamless scaling for their multi-tenant cluster architecture.Deploying a Serverless REST APIThis guide walks you through deploying a REST API using AWS services like API Gateway, Lambda, DynamoDB, and Cognito with Terraform. The project involves creating an API that allows users to manage a list of Sicilian dishes. It starts with configuring AWS as the provider and setting up an S3 bucket to store Terraform state files. You then create an IAM role with the necessary permissions for Lambda to interact with DynamoDB. The Lambda function itself is written in Python, with methods to handle CRUD operations on the DynamoDB table based on the incoming HTTP requests. Authentication is added via Amazon Cognito to secure write operations. Finally, the API routes (GET, POST, PATCH, DELETE) are implemented to handle the dish data, including a recursive scan function to fetch all dishes from the table.🔍Secret Knowledge: Learning ResourcesHow to Differentiate Manual Changes from Terraform Changes in S3 BucketTo differentiate manual changes from Terraform changes in an S3 bucket managed by Terraform, you can use AWS CloudTrail, EventBridge, Lambda, and SNS notifications. CloudTrail logs all S3 API actions, including manual and automated changes. EventBridge filters these logs for specific events (e.g., uploads or deletions) and triggers a Lambda function. The Lambda function processes the events to exclude actions initiated by Terraform (using the IAM role or userIdentity details associated with Terraform). It sends SNS notifications only for manual changes, ensuring Terraform modifications do not trigger alerts.Managing AWS EKS access entries with Terraform and OpenTofuManaging AWS EKS access entries with Terraform and OpenTofu simplifies authentication and authorization for Kubernetes clusters by replacing the outdated aws-auth ConfigMap with a more scalable and robust EKS API. Access entries allow direct API-based management of IAM users, roles, and predefined policies, eliminating manual ConfigMap updates prone to errors. With tools like Terraform and OpenTofu, you can define access entries as Infrastructure as Code (IaC), enabling automated and secure access control at scale. This method seamlessly integrates AWS IAM for authentication with Kubernetes RBAC for authorization.Uber’s billion trips migration setupUber successfully migrated its complex trip fulfillment infrastructure to a hybrid cloud environment without downtime by employing innovative strategies. To ensure uninterrupted service for millions of global users, Uber implemented a backward compatibility layer, maintaining support for existing APIs during the transition. They also used shadow validation, mirroring requests and comparing responses between old and new systems to identify discrepancies.30 Days of CNCF projects | Day 5: What is Crossplane + demoCI/CD automation with Tekton: GitHub PR pipeline guideAutomating CI/CD with Tekton involves leveraging its Kubernetes-native framework to define pipelines for building, testing, and deploying code. Using tools like Minikube, kubectl, and Ngrok, you can set up a local Tekton environment and integrate pre-built tasks such as git-clone and kaniko for cloning repositories and building Docker images. To automate workflows triggered by GitHub pull requests, Tekton Triggers can be configured to listen for webhooks, validate events, and execute pipelines, ensuring CI/CD tasks like building, testing, and updating GitHub statuses happen seamlessly.⚡TechWave:CloudNews & AnalysisKubernetes Gateway API v1.2: WebSockets, Timeouts, Retries, and MoreGateway API v1.2 introduces significant updates and improvements to Kubernetes networking, focusing on WebSocket support, HTTPRoute timeouts, retries, and more robust infrastructure annotations. It graduates several features, like HTTPRoute timeouts and backend protocol support, to the stable Standard channel, ensuring better resilience and backward compatibility. Notably, the release brings two breaking changes: the removal of outdated v1alpha2 versions for GRPCRoute and ReferenceGrant, and a shift in .status.supportedFeatures structure for greater future flexibility.Google Kubernetes Engine supports 65,000-node clustersGoogle Kubernetes Engine (GKE) now supports clusters of up to 65,000 nodes, a scale designed to meet the growing computational demands of massive AI workloads, including training and serving trillion-parameter AI models. This enhancement allows for faster training times, larger model scalability, and flexibility in resource allocation for diverse tasks. GKE achieves this through innovations like transitioning to a Spanner-based key-value store for enhanced reliability and a revamped control plane for faster scaling and operations.Container Insights with enhanced observability now available in Amazon ECSAmazon ECS now features enhanced observability with Container Insights, helping users monitor and troubleshoot container workloads more effectively. This capability offers detailed metrics, logs, and visual dashboards to quickly identify root causes of issues, reduce detection and repair times, and improve application performance. It supports granular resource monitoring, proactive issue management, cross-account observability, and seamless integration with CloudWatch services like Application Signals and Logs.New Amazon S3 Tables: Storage optimized for analytics workloadsAmazon S3 Tables are a new storage option optimized for analytics workloads, supporting tabular data in Apache Iceberg format. This managed service provides faster query performance (up to 3x) and handles higher transactions per second (up to 10x) compared to self-managed storage. S3 Tables integrate seamlessly with query engines like Amazon Athena and Apache Spark and include features like automatic maintenance (e.g., compaction, snapshot management) and logical grouping with namespaces.Grafana 11.4 release: Introducing support for OpenSearch PPL and OpenSearch SQL in the AWS CloudWatch data source pluginGrafana 11.4 now supports OpenSearch Piped Processing Language (PPL) and OpenSearch SQL in its AWS CloudWatch data source plugin, allowing AWS users more flexibility in querying CloudWatch Logs. These new query options join the existing Logs Insights QL, enabling users to filter and aggregate logs using their preferred language without duplicating data. Features like syntax highlighting, live code completion, and sample queries enhance usability.🛠️HackHub: Best Tools for CloudNovascans your cluster for installed Helm charts and then cross-checks them against all known Helm repositories.Pglite-fusionallows you to embed an SQLite database in your PostgreSQL table. AKA multitenancy has been solved.Drasiis a data processing platform that simplifies detecting changes in data and taking immediate action.SonarIACis a static code analyzer for Infrastructure-as-Code languages such as CloudFormation and Terraform, as well as DevOps like Docker and Kubernetes.Pg_flois a CLI to move and transform data between PostgreSQL databases using Logical Replication.📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

This Guide Helped Us Actually Save Money With Kubernetes Rightsizing- Start HereCloudPro #91: This Update Makes Kubernetes Smarter About Volume Limits Real stories from real engineers. How they blew their Azure budget on networking, what actually worked for IAM trust policies, how they enforced container image standards without killing velocity.That’s what this issue is about.🔐 Cloud SecurityThe One Thing You Should Never Skip When Accepting Public File UploadsHow to Keep IAM Trust Working. Without Breaking on Role RecreationNo One Tells You This About Building Cybersecurity Products: You Become the TargetWhat a Real Secure GenAI Deployment Looks Like on AWSThe One Thing You Should Never Do When Building a Platform: Rely on Tickets, Raw Cloud, or Rigid Templates⚙️ Infrastructure & DevOpsHow AWS Fixed the Hardest Part of Data Lake ManagementHow We Enforced Image Standards in ECS Without Slowing Down DevsWe Blew Our Azure Budget on Networking- Here’s the Guide I Wish We HadHow a Simple Misconfig Let Us Pull Images from Any S3 Bucket. No CVE NeededIf You’re Still Writing IaC by Hand, Read This Firstgit-whois a command-line tool for answering that eternal question: Who wrote this code?📦 Kubernetes & Cloud NativeThis Update Makes Kubernetes Smarter About Volume LimitsIf You’re Still Treating Sidecars Like Init Containers, You’re Doing It WrongThis Guide Helped Us Actually Save Money With Kubernetes Rightsizing- Start HereWhat a Real-World Ingress Setup Looks Like on AWS EKSDocker Engine v28: Hardening container networking by default🔍 Observability & SREGrafana 12 release: observability as code, dynamic dashboards, new Grafana Alerting tools, and more | Grafana LabsCloudWatch Dashboard (Over)Sharing | WithSecure™ LabsPrezi's Journey from Prometheus to VictoriaMetrics - InfoQOpenTelemetry: A Guide to Observability with Go | Blog | Luca CavallinKHIis a rich log visualization tool for Kubernetes clusters. KHI transforms vast quantities of logs into an interactive, comprehensive timeline view.Cheers,Shreyans SinghEditor-in-ChiefNew developer products provide a glimpse into the future of app building on HubSpot, including deeper extensibility, flexible UI, modern development tools, and moreHubSpot’s AI-powered ecosystem presents a global opportunity projected to reach $10.2 billion by 2028.To capitalize on that growth potential, we are opening our platform more, starting with expanded APIs, customizable app UI, and tools that better support a unified data strategy.Start Building TodayForward to a Friend📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

PostgreSQL cloud native High Availability and moreCloudPro #72: Better Kubernetes YAML Editing with (Neo)vim⭐Masterclass:Unlock the Full Potential of Kubernetes for Scalable Application ManagementKubernetes pod and container restartingBetter Kubernetes YAML Editing with (Neo)vimMonitoring kubernetes events with kubectl and Grafana LokiPractical Logging for PHP Applications with OpenTelemetryUsing 1Password with External Secrets Operator in a GitOps way🔍Secret Knowledge:Build your own SQS or Kafka with PostgresRevealing the Inner Structure of AWS Session TokensAn Opinionated Ramp Up Guide to AWS PentestingGang scheduling pods on Amazon EKS using AWS Batch multi-node processing jobsApplication Availability Depends on Dependencies🛠️HackHub: Best Tools for the CloudPostgreSQL cloud native High Availability and moreKubernetes Operator to automate Helm, DaemonSet, StatefulSet & Deployment updatesRuns and manages databases, message queues, etc on K8sPowerful workflow engine and end-to-end pipeline solutions implemented with native Kubernetes resourcesconfigure kubernetes objects on multiple clusters using jsonnetCheers,Shreyans SinghEditor-in-Chief⭐MasterClass: Tutorials & GuidesKubernetes pod and container restartingIn Kubernetes, a Pod is the smallest deployable unit, often containing one or more containers. When a container or pod needs to be restarted due to errors or updates, Kubernetes offers several methods to do so. For example, you can restart a Pod by deleting it, and Kubernetes will automatically recreate it if it’s part of a Deployment. Alternatively, you can restart a specific container within a Pod using commands like `kubectl exec` for more precise control. These features allow Kubernetes to maintain high availability and resilience in a cloud environment.Better Kubernetes YAML Editing with (Neo)vimEditing Kubernetes YAML files can be tricky, but using Neovim, a modern version of Vim, can make it much easier. Neovim is lightweight, highly customizable, and integrates well with your terminal, making it ideal for DevOps and platform engineers. By configuring Neovim specifically for YAML files, you can set up features like auto-indentation, syntax highlighting, folding, and autocompletion, all of which help reduce errors and improve efficiency.Monitoring kubernetes events with kubectl and Grafana LokiIn Kubernetes, monitoring events is crucial for understanding the status and issues related to Pods, WorkerNodes, and other components. You can use `kubectl` to view these events directly, or you can enhance your monitoring setup by integrating Kubernetes events with Grafana Loki. By capturing events as logs using a tool like the `k8s-event-logger`, which listens to the Kubernetes API, you can store them in Loki, create metrics with RecordingRules, and visualize them in Grafana.Practical Logging for PHP Applications with OpenTelemetryPractical logging for PHP applications using OpenTelemetry involves instrumenting your PHP code to collect and correlate log data with other observability signals like traces and metrics. This approach is particularly useful in microservices-based architectures, where understanding the interactions between different services is crucial for maintaining system stability. By using OpenTelemetry, developers can standardize how telemetry data is collected and exported, reducing complexity.Using 1Password with External Secrets Operator in a GitOps wayTo manage secrets securely in a GitOps environment using Kubernetes, you can integrate 1Password with the External Secrets Operator. This setup allows you to automatically fetch and inject secrets stored in 1Password into your Kubernetes cluster. By using tools like ArgoCD, Helm, or FluxCD, you can deploy and manage this integration efficiently. The External Secrets Operator pulls secrets from 1Password via 1Password Connect, a proxy that ensures availability and reduces API requests.🔍Secret Knowledge: Learning ResourcesBuild your own SQS or Kafka with PostgresYou can build your own version of SQS (Simple Queue Service) or Kafka using PostgreSQL by setting up tables and queries that mimic the functionality of these popular message queues and streams. For SQS, you create a table to store messages, with columns that help manage message visibility, delivery attempts, and order. You can then write queries to insert messages, retrieve them while respecting visibility timeouts, and delete them after processing. For Kafka, you expand this setup by storing messages persistently and keeping track of where each consumer group is in the message stream, allowing multiple consumers to process messages independently and in parallel, similar to Kafka's partitioning system.Revealing the Inner Structure of AWS Session TokensBy reverse engineering these tokens, the research team developed tools to analyze and modify them programmatically. This allowed them to uncover previously unknown details about AWS's cryptography and authentication protocols. Their findings showed that while AWS's security measures are robust, understanding the structure of these tokens can help defenders better protect against potential attacks. Additionally, the research raises questions about the privacy and integrity of these tokens.An Opinionated Ramp Up Guide to AWS Pentesting)Lizzie Moratti's "Opinionated Ramp Up Guide to AWS Pentesting" offers a detailed roadmap for becoming proficient in AWS pentesting, emphasizing practical experience over certifications. The guide is tailored for those with a foundational understanding of networking and security, and it stresses the importance of broad knowledge before delving into deeper cloud-specific skills. The guide also touches on industry pitfalls, such as reliance on automated tools and the challenges of cloud pentesting in a fast-evolving environment.Gang scheduling pods on Amazon EKS using AWS Batch multi-node processing jobsAWS Batch now supports multi-node parallel (MNP) jobs for Amazon EKS, allowing you to gang schedule pods across multiple nodes for tasks that require extensive computation, like machine learning or weather forecasting. Previously, MNP jobs were only available on Amazon ECS. With this update, you can use AWS Batch on EKS to run distributed processing jobs, such as those with Dask, a Python library for parallel computing. The setup involves defining job configurations that include a main node running the scheduler and worker nodes executing the tasks. This approach ensures efficient communication and scaling across nodes, streamlining complex computations in a managed environment.Application Availability Depends on DependenciesModern applications depend on various services and components, meaning their reliability is tightly linked to the uptime of these dependencies. For example, if an application like Tekata.io needs to maintain 99.9% uptime, but it relies on several services with only 99.9% uptime each, the combined effect could reduce Tekata.io’s overall availability. To hit the desired uptime, dependencies need to have even higher availability. The formula \( A = U^N \) shows that if your application’s target uptime is 99.9% and it has 7 dependencies, each dependency must have an uptime of 99.99% to meet that target.🛠️HackHub: Best Tools for Cloudsorintlab/stolonStolon is a cloud-native tool designed to manage PostgreSQL databases with high availability, making it suitable for deployment in various environments including Kubernetes and traditional infrastructures. It leverages PostgreSQL's streaming replication and integrates with cluster stores like etcd, Consul, or Kubernetes for leader election and data storage.keel-hq/keelKeel is a lightweight tool for automating updates to Kubernetes deployments without needing complex command-line interfaces or APIs. It integrates directly with Kubernetes and Helm, using labels and annotations to manage updates based on semantic versioning policies.apecloud/kubeblocksKubeBlocks is an open-source tool designed to simplify the management of multiple database types on Kubernetes using a unified set of APIs. Instead of dealing with different operators for each database, KubeBlocks provides a single control plane to manage various databases such as PostgreSQL, Redis, and Kafka. It offers a standardized approach to database lifecycle management, day-2 operations, and observability, with support for backup, recovery, and monitoring.caicloud/cycloneCyclone is a workflow engine built for Kubernetes that manages end-to-end pipelines without requiring extra dependencies. It operates across various Kubernetes environments, including public, private, and hybrid clouds. Cyclone offers features like DAG graph scheduling, flexible parameterization, and integration with external systems. It supports triggers, multi-cluster execution, multi-tenancy, and automatic resource cleanup.splunk/qbecQbec is a CLI tool designed for managing Kubernetes objects across multiple clusters or namespaces using jsonnet, a data-templating language. It simplifies Kubernetes configuration management by allowing users to define and deploy objects in various environments efficiently. Qbec is similar to tools like kubecfg and ksonnet.📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}.reverse{display:table;width: 100%;

CloudPro Special EditionCloudPro Special: Mastering Serverless Architecture- A Concise GuideAfter the last CloudPro Special, a lot of subscribers messaged me that I should do it more often. So here we are, with the next CloudPro Special. Today, I’ll talk about Serverless Architecture.There are eight sections:1. Introduction to Serverless Architecture2. Designing Serverless Solutions3. Using Serverless with AWS4. Serverless in the Bigger Picture5. Real-world Use Cases6. Monitoring Serverless Apps7. Pros and Cons8. Conclusion`Each section has additional learning resources:Cloud Computing Demystified for Aspiring ProfessionalsArchitecting Cloud-Native Serverless SolutionsMulti-Cloud Strategy for Cloud ArchitectsAWS CDK in PracticeMulti-Cloud Handbook for DevelopersAWS for Solutions ArchitectsMastering Amazon EC2Solutions Architect's HandbookAWS Certified Developer Associate Certification and BeyondAWS Observability HandbookAny feedback or questions, just reply back to this email and let me know. Without further delay, let's jump into today's CloudPro Special!Cheers,Shreyans SinghEditor in ChiefIntroduction to Serverless ArchitectureServerless architecture is a new way to build and run apps without worrying about servers. Despite its name, there are still servers involved, but cloud providers like Amazon or Google manage them for you. This means you can focus on writing code instead of managing hardware.Serverless is the latest step in cloud computing. It started with physical machines, then moved to services where you rent virtual servers, and now we have serverless where you just run your code without thinking about the servers at all.💡Learning ResourceTo learn more about serverless and cloud computing, check out "Cloud Computing Demystified for Aspiring Professionals" ($24.99 $35.99). It explains these concepts in simple terms.Learn more about cloud computing and serverlessDesigning Serverless SolutionsWhen building serverless apps, keep these things in mind:📌Make your functions independent (they shouldn't rely on saved information)📌Build your app around events and triggers📌Break your app into small, separate services📌Be aware of "cold starts" which can slow things down📌Make sure your functions finish quickly (there are time limits)Remember, serverless isn't always the cheapest option. It works best for apps with unpredictable usage. For apps with steady, predictable usage, traditional servers might be cheaper.💡Learning ResourceTo learn more about designing serverless apps, I recommend "Architecting Cloud-Native Serverless Solutions" ($24.99 $35.99). It's a helpful guide for building serverless apps on different cloud platforms.Learn more about designing serverless solutionsIf you want to use serverless with multiple cloud providers, check out "Multi-Cloud Strategy for Cloud Architects" ($29.99 $43.99). It helps you understand how to use serverless across different providers.Learn about using multiple cloud providersUsing Serverless with AWSAWS offers many serverless tools. Here are the main ones:📌AWS Lambda: Run your code📌Amazon API Gateway: Create and manage APIs📌AWS Step Functions: Coordinate multiple functions📌Amazon EventBridge: Build apps that respond to events📌AWS SAM: Make serverless development easierOne useful tool for deploying serverless apps on AWS is the AWS Cloud Development Kit (CDK). It lets you set up your cloud infrastructure using regular programming languages.💡Learning ResourceTo learn how to use AWS CDK, I suggest "AWS CDK in Practice" ($27.98 $39.99). It teaches you how to build complex serverless apps easily.Learn about AWS CDKTo compare serverless options across different providers, check out "Multi-Cloud Handbook for Developers" ($27.98 $39.99).Learn about different cloud providersServerless in the Bigger PictureWhile serverless is great, it's not perfect for every situation. Sometimes, traditional cloud services might work better. For example, if you have long-running processes or steady workloads, using regular servers or containers might be better.To understand when to use serverless and when to use other options, we recommend these resources:💡Learning ResourceLearn about AWS architectureLearn about traditional cloud computingLearn about different cloud architecturesReal-world Use CasesMany companies use serverless successfully. Here are a few examples:📌Coca-Cola: Used serverless for vending machines, cutting costs by 65%.📌Netflix: Uses serverless for tasks like processing videos, handling billions of events daily.📌Zalora: Switched to serverless, reducing infrastructure costs by 60%.These companies learned some important lessons:📌Start small and gradually move more parts of your app to serverless📌Use good monitoring tools📌Use code to manage your infrastructure📌Keep improving your functions' performance and cost💡Learning ResourceTo learn more about real-world serverless uses, check out "AWS Certified Developer Associate Certification and Beyond" ($27.98$39.99). It gives practical insights into building serverless solutions on AWS.Learn about serverless development on AWSMonitoring Serverless AppsKeeping an eye on serverless apps is different from traditional apps. You need to watch:📌How different functions work together📌How long it takes for functions to start and run📌How much your functions cost to run📌Logs from all your functions💡Learning ResourceTo learn how to monitor serverless apps on AWS, I recommend the "AWS Observability Handbook" ($27.98 $39.99) It shows you how to use AWS tools to watch, track, and get alerts about your serverless apps.Learn about AWS monitoringPros:📌Can be cheaper for some types of apps📌Automatically handles more users📌Lets you release new features faster📌Lets developers focus on code📌No need to manage serversCons:📌Can be slow to start sometimes📌Limited run time (usually max 15 minutes)📌Can be harder to find and fix problems📌Might be hard to switch to a different provider📌Less control over the underlying systemConclusionServerless is a powerful way to build apps that can handle any number of users and potentially save you money. It lets developers focus on writing code instead of managing servers.Remember, serverless isn't always the best choice. Always consider your specific needs when deciding whether to use serverless.To learn more, check out the learning resources mentioned in this guide. They'll help you become an expert in serverless computing.Happy learning, and good luck with your serverless projects!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{line-height:0;font-size:75%} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Cloudflare adopted OpenTelemetry for logging pipeline CloudPro #71: A hard look at GuardDuty shortcomings ⭐Masterclass: From Docker Compose to Kubernetes Manifests A hard look at GuardDuty shortcomings Streamlining Keycloak in Kubernetes The hater’s guide to Kubernetes A skeptic's first contact with Kubernetes 🔍Secret Knowledge: Enhancing Bitnami Helm Charts Security Cloudflare adopted OpenTelemetry for logging pipeline Josh Grose on LinkedIn: I spent the last 3 yrs outside of observability Did you know the CNCF has an actual cookbook? Not metaphorically! Unfashionably secure: why we use isolated VMs 🛠️HackHub: Best Tools for the Cloud Web tool for database management The devs are over here at devzat, chat over SSH! CloudFormation_To_Terraform Debugging tool for Kubernetes which tests and displays connectivity between nodes in the cluster. Kubernetes network solution Cheers, Shreyans Singh Editor-in-Chief Forward to a Friend 🔍Secret Knowledge: Learning Resources Related Titles Enhancing Bitnami Helm Charts Security Bitnami enhanced the security of its Helm charts using Kubescape, an open-source Kubernetes security tool that identifies misconfigurations by comparing configurations to industry best practices. By integrating Kubescape into their build pipelines, Bitnami made significant improvements such as eliminating group root dependencies, configuring immutable filesystems, and reducing misconfigured resources. Cloudflare adopted OpenTelemetry for logging pipeline Cloudflare recently transitioned its logging pipeline from syslog-ng to OpenTelemetry Collector to enhance performance, maintainability, and telemetry insights. This move allowed the team to leverage Go, a language more familiar to their engineers, and integrate better observability through Prometheus metrics. Despite challenges like minimizing downtime during the switch and ensuring compatibility with existing infrastructure, the migration has opened up opportunities for further improvements, such as better log sampling and migration to the OpenTelemetry Protocol (OTLP). Josh Grose on LinkedIn: I spent the last 3 yrs outside of observability Josh Grose (ex-Principal PM, Splunk), after three years away from the observability space, was surprised to find that despite companies spending around 30% of their cloud budgets on monitoring, reliability hasn't improved significantly. He observed that even when Service Level Agreements (SLAs) are met, it often comes at the cost of developer productivity and experience. Engineering leaders are frustrated with the high costs and limited improvements in key metrics like Mean Time to Recovery (MTTR) and development speed, leading to the perception that observability has become an expensive and ineffective necessity. Did you know the CNCF has an actual cookbook? Not metaphorically! The "Cloud Native Community Cookbook" is a unique collection of recipes put together by the CNCF and Equinix Metal, born out of the increased time people spent at home during the COVID-19 pandemic. Instead of focusing on cloud technologies, this cookbook brings together food recipes shared by members of the Cloud Native community, originally exchanged in Equinix Metal's Slack channel. Unfashionably secure: why we use isolated VMs While modern cloud architectures often favor shared, multi-tenant environments for efficiency and scalability, Thinkst Canary opts for a less trendy but highly secure approach by using isolated virtual machines (VMs) for each customer. This choice prioritizes security by ensuring that each customer's data and services are completely separated, reducing the risk of cross-customer data breaches. Although this method comes with higher operational costs and complexity, it provides a stronger security boundary, making it easier to manage risks and sleep better at night. ⚡TechWave: Cloud News & Analysis How Figma Migrated onto K8s in Less Than 12 months Figma completed its migration to Kubernetes in under a year by meticulously planning and executing a well-scoped transition. Initially running services on AWS's ECS, Figma faced limitations such as complex stateful workloads and limited auto-scaling. The decision to move to Kubernetes (EKS) was driven by its broader functionality, including support for StatefulSets, Helm charts, and advanced scaling options from the CNCF ecosystem. By Q1 2024, Figma had migrated most core services with minimal impact on users, resulting in enhanced reliability, reduced costs, and a more flexible compute platform. Github Copilot Autofix: Secure code 3x faster Copilot Autofix, now available in GitHub Advanced Security, is an AI-powered tool designed to help developers fix code vulnerabilities more than three times faster than manual methods. It analyzes vulnerabilities, explains their significance, and offers code suggestions for quick remediation. This accelerates the fixing process for both new vulnerabilities and existing security debt, significantly reducing the time and effort required for secure coding. Copilot Autofix is included by default for GHAS customers and also available for open source projects starting in September. New Kubernetes CPUManager Static Policy: Distribute CPUs Across Cores Kubernetes v1.31 introduces a new alpha feature called "distribute-cpus-across-cores" for the CPUManager's static policy. This option aims to enhance performance by spreading CPUs more evenly across physical cores, rather than clustering them on fewer cores. This reduces contention and resource sharing between CPUs on the same core, which can boost performance for CPU-intensive applications. To use this feature, users need to adjust their Kubernetes configuration to enable it. Currently, it cannot be combined with other CPUManager options, but future updates will address this limitation. Announcing mandatory multi-factor authentication for Azure sign-in Microsoft is making multi-factor authentication (MFA) mandatory for all Azure sign-ins to enhance security and protect against cyberattacks. Starting in the latter half of 2024, Azure users will need to use MFA to access the Azure portal and admin centers, with broader enforcement for other Azure tools like CLI and PowerShell set for early 2025. MFA, which adds an extra layer of security by requiring more than just a password, is shown to block over 99% of account compromises. GitHub scales on demand with Azure Functions GitHub faced scalability issues with its internal data pipeline, which struggled to handle the massive amount of data it collects daily. To address this, GitHub partnered with Microsoft to use Azure Functions' new Flex Consumption plan, which allows serverless functions to scale dynamically based on demand. This solution has enabled GitHub to efficiently process up to 1.6 million events per second, addressing their growth challenges and improving performance with minimal overhead. 🛠️HackHub: Best Tools for Cloud commandprompt/pgmanage PgManage is a modern graphical database client for PostgreSQL, focusing on management features and built on the now-dormant OmniDB project. quackduck/devzat Devzat is a chat service accessible via SSH that replaces the traditional shell prompt with a chat interface, allowing you to connect from any device with SSH capabilities. aperswal/CloudFormation_To_Terraform The CloudFormation to Terraform Converter is a tool that automates the migration of AWS CloudFormation templates to Terraform configuration files. bloomberg/goldpinger Goldpinger monitors Kubernetes networking by making calls between its instances and providing Prometheus metrics for visualization and alerts. ZTE/Knitter Knitter is a Kubernetes CNI plugin that supports multiple network interfaces for pods, allowing custom network configurations across various cloud environments. 📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us. If you have any comments or feedback, just reply back to this email. Thanks for reading and have a great day! *{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Designing Scalable Microservices with KubernetesCloudPro: Special IssueHey there,As cloud professionals, we are always looking for ways to improve our skills and build solutions that are scalable, secure, and efficient. While regular news and updates keep us informed, sometimes it's good to take a deep dive into topics that matter.That’s why we’re bringing you this special issue of CloudPro, where we explore two carefully selected books that provide practical, hands-on learning experiences.The first book, AWS Cloud Projects, takes a step-by-step approach to building real-world cloud solutions. We’ll walk through a key project—Deploying a Serverless Application on AWS Lambda with Terraform—to help you understand infrastructure as code (IaC) in action.The second book, Solutions Architect’s Handbook, goes beyond the basics to teach cloud-native architecture best practices. We’ve included an in-depth section on Designing Scalable Microservices with Kubernetes to help you optimize your deployments for performance and efficiency.If you’re serious about learning by doing, this issue is for you. Dive in and explore!AWS Cloud ProjectsLately, I've been thinking a lot about the value of hands-on learning. There's something about actually building projects that sticks with you far longer than just reading concepts. That’s why when I came across AWS Cloud Projects, I knew it was worth sharing with you.This book doesn’t just explain AWS concepts—it walks you through real-world implementations, step by step. Whether you’re spinning up cloud infrastructure, deploying AI-powered applications, or optimizing security, the projects in this book serve as practical blueprints.One particular chapter stood out: Deploying a Serverless Application on AWS Lambda with Terraform. Here’s a detailed excerpt to give you a strong foundation:"In this project, we’ll set up a serverless API using AWS Lambda and API Gateway, all provisioned through Terraform. Infrastructure as Code (IaC) allows us to automate deployments, ensuring repeatability and reducing manual effort.Step 1: Define the Lambda FunctionWe start by defining our Lambda function using Terraform. Below is a basic Terraform configuration to deploy a function:resource "aws_lambda_function" "my_lambda" { function_name = "serverless_api" handler = "index.handler" runtime = "nodejs14.x" role = aws_iam_role.lambda_exec.arn filename = "lambda.zip"}Step 2: Configure API GatewayAPI Gateway allows our Lambda function to be exposed as an HTTP endpoint:resource "aws_api_gateway_rest_api" "api" { name = "serverless_api" description = "API Gateway for our Lambda function"}Step 3: Deploying the InfrastructureTo apply these changes, we use:terraform initterraform apply -auto-approveBy following these steps, you’ll have a fully operational serverless API deployed on AWS using Terraform.If you're someone who learns best by building, AWS Cloud Projects is a must-read. It’s the kind of book that makes learning AWS both practical and engaging.GET IT HERESolutions Architect's HandbookCloud-Native Architecture: Scaling Beyond LimitsAnother book I recently found valuable is Solutions Architect’s Handbook. It goes beyond the basics of cloud architecture and explores scalability, security, and generative AI in real-world applications. If you’re serious about designing scalable cloud systems, this book is a gem.Here’s a deep dive into Designing Scalable Microservices with Kubernetes:"When designing microservices at scale, Kubernetes provides a resilient, self-healing platform. But to truly optimize performance, we must consider three key factors: resource allocation, observability, and network efficiency.Resource Optimization with Horizontal Pod Autoscaler (HPA)Using HPA ensures workloads dynamically adjust to demand:apiVersion: autoscaling/v2kind: HorizontalPodAutoscalermetadata: name: my-app-hpaspec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: my-app minReplicas: 2 maxReplicas: 10 metrics: - type: Resource resource: name: cpu targetAverageUtilization: 50Observability with Prometheus & GrafanaMonitoring plays a crucial role in scaling applications. The book explains how to integrate Prometheus and Grafana for real-time insights into resource consumption and request rates.apiVersion: monitoring.coreos.com/v1kind: ServiceMonitormetadata: name: my-app-monitorspec: selector: matchLabels: app: my-app endpoints: - port: metricsNetwork Optimization using IstioIstio allows fine-grained traffic control, helping balance workloads efficiently.apiVersion: networking.istio.io/v1alpha3kind: VirtualServicemetadata: name: my-appspec: hosts: - "my-app.example.com" http: - route: - destination: host: my-app subset: v1With these techniques in place, you can scale workloads efficiently without unnecessary costs. Solutions Architect’s Handbook provides a deeper look into how enterprises design cloud-native applications for high availability and scalability.GET IT HERE📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}