[{"content":"","date":null,"permalink":"/tags/arc42/","section":"Tags","summary":"","title":"Arc42"},{"content":"","date":null,"permalink":"/categories/architecture/","section":"Categories","summary":"","title":"Architecture"},{"content":"","date":null,"permalink":"/categories/","section":"Categories","summary":"","title":"Categories"},{"content":"","date":null,"permalink":"/categories/culture/","section":"Categories","summary":"","title":"Culture"},{"content":"The Isolation Pattern #Engineers are trained to execute. You get user stories, requirements, and acceptance criteria, then deliver against them. The system should handle X concurrent users, scale to Y nodes, and respond in under Z milliseconds. Those are real constraints, and meeting them takes serious work.\nBut here is the question nobody asks enough: how many devices in the field are actually running the latest version, and how many users actively use what we built? What does success look like for the product, not just for the current sprint?\nWhen engineers do see those numbers, the conversation changes immediately. The problem is structural: product owns the outcome metrics, engineering owns the system metrics, and neither dashboard talks to the other. Both sides are working hard, and both believe they delivered. When the outcome disappoints, neither can easily explain why, because success was never defined in the same room.\nArc42 is a widely adopted template for documenting software architecture, structured around twelve sections covering everything from system scope to deployment and quality requirements. Section 1 (Introduction and Goals) asks exactly the right questions—specifically around quality goals and stakeholders1. But in practice, engineers fill them in based on what they understood from the requirements, not from a joint conversation about what the product needs to achieve. It is not that the framework is missing anything. It is a habit: the sections get written alone, and they stay that way.\nDefine the Measure Before You Build #Lean manufacturing solved this decades ago: you cannot inspect quality into a product at the end of the line; you must build it into the process from the start2. The same applies to software. Scaling to ten nodes and handling a thousand requests per second is not the same as knowing whether a feature is used, whether it reduced manual work, or whether a device fleet in the field is actually running the latest version. The most common failure mode here is a system that passes every test and hits every SLA, but whose impact is either invisible, because nobody instrumented it, or quietly categorized as a cost by leadership, because the connection to business outcomes was never made explicit. The metrics that prevent this do not need to be complicated: adoption rate, time saved per task, reduction in support tickets, devices successfully updated in the field, active users per feature. Simple, concrete measures that both sides agree on before a ticket is written.\nI\u0026rsquo;ve found Impact Mapping to be the most effective way to stop \u0026ldquo;feature factory\u0026rdquo; behavior3, but it is not purely an engineering exercise—it is a product exercise that engineering needs to be part of. Product traces the path from the business goal (Why), to the user (Who), to the desired behavioral change (How). Engineering\u0026rsquo;s role is to ground that map in technical reality: to validate which deliverables (What) are feasible, where instrumentation is missing, and whether the proposed features can actually be measured once shipped. If a feature cannot be traced back to a behavioral change on that map, it should not reach the backlog.\nOnce those impacts are mapped jointly, they can be translated into concrete engineering requirements using the Goal-Question-Metric (GQM) framework4. The model is simple but forces discipline:\nDefine a Goal (e.g., improve onboarding completion across new device types). Ask the Questions that tell you whether the goal is met (e.g., where do users drop off during provisioning?). Instrument the precise Metrics that answer those questions (e.g., telemetry on step-completion latency per device class). Telemetry is not an afterthought, if a feature cannot be measured, it is not complete. Arc42 Section 10 (Quality Requirements) is built around concrete Quality Scenarios for exactly this reason. The missing step is including product in that conversation: one focused session before implementation starts, not a new recurring ceremony.\nClosing the Loop #Defining metrics upfront only matters if you come back to them. Once the system is running, use one of the meetings you already have — a retrospective, a quarterly review, whatever fits — and check whether outcomes materialized. Did adoption go up? Did manual steps decrease? This does not need its own ceremony; it just needs someone to bring the numbers.\nI learned this the hard way when a technically successful project I led was canceled during a political reshuffle. We had the metrics, but because they weren\u0026rsquo;t part of the \u0026ldquo;success\u0026rdquo; conversation with leadership from day one, our work was viewed solely as a cost center. By the time we presented the value in the post-mortem, the decision was already made. Technical excellence doesn\u0026rsquo;t save a project if the business doesn\u0026rsquo;t know how to measure its value.\nWhen closing an iteration, the feature showcase is part of it — but it does not stop there. Alongside what we built, we present what moved: edge devices deployed across fourteen countries, X updates pushed to the fleet, Y bug reports received, deployment time down from X to Y minutes, operator touchpoints cut in half. That context is what turns a status report into a decision. \u0026ldquo;We closed twelve tickets\u0026rdquo; tells the room nothing. \u0026ldquo;We closed twelve tickets, the automation is working, but the failure pattern is still regional\u0026rdquo; tells the room exactly where to focus next.\nTo scale this without manual overhead, these checks can be automated in CI pipelines using Architectural Fitness Functions5 — ensuring the system continuously validates the quality scenarios and business metrics defined at the start, not just at the end of an iteration.\nConclusion #The gap between what engineering delivers and what the business values is a coordination problem, and it compounds quietly. The tools to fix it already exist — Arc42 has the sections, Impact Mapping has the structure, GQM has the discipline. What is missing is the habit of using them together, with the right people in the room, before a single ticket is written.\nNext sprint: before the backlog is groomed, get product in the room to agree on what success looks like. Write it in Section 1. Define three measurable quality scenarios in Section 10 together. Then close the iteration with those numbers, not a feature list.\nThat is the difference between engineering as a cost and engineering as an investment. And it is entirely within your control.\narc42 Chapter 1: Introduction and Goals — Michaël\u0026rsquo;s Coding Thoughts\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nWhat Is Lean Product Engineering? A Practical Playbook — LinearLoop\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nHow Impact Mapping Empowers Cloud Architects — Christian Bonzelet (cremich.cloud)\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nUnderstanding the GQM Approach — Victor Basili \u0026amp; David Weiss, Scribd\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nArchitectural Fitness Function — Thoughtworks Technology Radar\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"May 17, 2026","permalink":"/posts/engineering-outcomes-shared-responsibility/","section":"Posts","summary":"\u003ch2 id=\"the-isolation-pattern\" class=\"relative group\"\u003eThe Isolation Pattern \u003cspan class=\"absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100\"\u003e\u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700\" style=\"text-decoration-line: none !important;\" href=\"#the-isolation-pattern\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\u003c/span\u003e\u003c/h2\u003e\u003cp\u003eEngineers are trained to execute. You get user stories, requirements, and acceptance criteria, then deliver against them. The system should handle X concurrent users, scale to Y nodes, and respond in under Z milliseconds. Those are real constraints, and meeting them takes serious work.\u003c/p\u003e\n\u003cp\u003eBut here is the question nobody asks enough: how many devices in the field are actually running the latest version, and how many users actively use what we built? What does success look like for the product, not just for the current sprint?\u003c/p\u003e","title":"Engineering Beyond Delivery: Shared Responsibility for Outcomes"},{"content":"I build secure, cloud-native platforms across distributed environments,from embedded Linux to Kubernetes fleet management at scale.\n","date":null,"permalink":"/","section":"Engineering Infrastructure as a Product","summary":"\u003cp\u003eI build secure, cloud-native platforms across distributed environments,from embedded Linux to Kubernetes fleet management at scale.\u003c/p\u003e","title":"Engineering Infrastructure as a Product"},{"content":"","date":null,"permalink":"/tags/platform-engineering/","section":"Tags","summary":"","title":"Platform Engineering"},{"content":"","date":null,"permalink":"/posts/","section":"Posts","summary":"","title":"Posts"},{"content":"","date":null,"permalink":"/tags/product-management/","section":"Tags","summary":"","title":"Product Management"},{"content":"","date":null,"permalink":"/tags/","section":"Tags","summary":"","title":"Tags"},{"content":"","date":null,"permalink":"/tags/infrastructure/","section":"Tags","summary":"","title":"Infrastructure"},{"content":"In the world of engineering, we often talk about \u0026ldquo;systems\u0026rdquo; and \u0026ldquo;products,\u0026rdquo; but the real differentiator for scaling is the Platform. As teams grow and systems become more complex, shifting from manual setup to a structured, product-centric approach is essential.\nImage by Pexels from Pixabay\nDefining the Platform: Infrastructure as a Product #A platform is more than just a set of tools. It is a reliable way to ship software that handles the difficult parts of the infrastructure for you.\nInstead of every team having to figure out how to set up hardware or the operating system from scratch, the platform provides a proven template that just works. This is what we call Infrastructure as a Product.\nToday, Kubernetes has become the standard engine for these platforms. It provides a common language and a set of automated tools that treat your foundation with the same care as a customer-facing feature. By providing ready-to-use patterns and automated safety checks, the platform lets developers focus on their actual work.\nOne of the most important aspects of a platform is that it is shared across different teams. Instead of every team building their own unique way of working, everyone uses the same proven foundation. This means:\nShared Standards: Everyone speaks the same technical language, making it easier to collaborate and move between projects. Shared Improvements: When the platform is updated or a new safety check is added, every team using it gets those benefits immediately. No More \u0026ldquo;Reinventing the Wheel\u0026rdquo;: Teams stop wasting time solving the same infrastructure problems and spend more time building their specific product features. The infrastructure then handles the rest:\nRepeatability: Because it uses Kubernetes, your software works exactly the same way every time, whether you are in the cloud or at a remote site. Autonomy: Kubernetes is designed to handle itself. It constantly checks that your system matches your \u0026ldquo;source of truth.\u0026rdquo; It can fix common issues or restart failed parts without needing a person to manually intervene. Governance: The \u0026ldquo;rules of the house\u0026rdquo; are built directly into the system. Standards and safety checks happen automatically as part of the normal workflow. How the Platform Works #To build a platform that actually delivers these benefits—especially at the edge—we focus on three main areas:\nA Hardened Foundation: We create a minimal base that includes only what is strictly necessary. We then run a lightweight version of Kubernetes (like k3s) on top of it. This makes the system faster, simpler, and much safer. Automated State Management: Kubernetes is built to manage \u0026ldquo;state.\u0026rdquo; We define how the entire system should look in a central place. The platform then ensures that every part of the infrastructure stays aligned with that \u0026ldquo;source of truth.\u0026rdquo; Reliable, Atomic Updates: We ensure that updates are \u0026ldquo;all or nothing.\u0026rdquo; A change either finishes completely or it doesn\u0026rsquo;t happen at all. This prevents the system from getting stuck in a broken state. Conclusion #Building a platform means creating a path that allows teams to move fast without breaking things. By treating infrastructure as a product, we can provide a foundation that is reliable, secure, and ready to scale.\n","date":"May 10, 2026","permalink":"/posts/what-is-a-platform/","section":"Posts","summary":"\u003cp\u003eIn the world of engineering, we often talk about \u0026ldquo;systems\u0026rdquo; and \u0026ldquo;products,\u0026rdquo; but the real differentiator for scaling is the \u003cstrong\u003ePlatform\u003c/strong\u003e. As teams grow and systems become more complex, shifting from manual setup to a structured, product-centric approach is essential.\u003c/p\u003e\n\u003cp\u003e\n\n\n\n\n\n\n  \n  \n\u003cfigure\u003e\u003cimg src=\"/img/concept-1868728_640.jpg\" alt=\"platform-concept\" class=\"mx-auto my-0 rounded-md\" /\u003e\n\u003c/figure\u003e\n\n\u003cem\u003eImage by Pexels from Pixabay\u003c/em\u003e\u003c/p\u003e\n\u003ch2 id=\"defining-the-platform-infrastructure-as-a-product\" class=\"relative group\"\u003eDefining the Platform: Infrastructure as a Product \u003cspan class=\"absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100\"\u003e\u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700\" style=\"text-decoration-line: none !important;\" href=\"#defining-the-platform-infrastructure-as-a-product\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\u003c/span\u003e\u003c/h2\u003e\u003cp\u003eA platform is more than just a set of tools. It is a \u003cstrong\u003ereliable way to ship software\u003c/strong\u003e that handles the difficult parts of the infrastructure for you.\u003c/p\u003e","title":"What is a Platform? Infrastructure as a Product"},{"content":"Over the past decade1, numerous high-impact vulnerabilities, such as Heartbleed (2014), WannaCry (2017), SolarWinds ( 2020), and Log4Shell (2021)2. Have exposed systemic flaws in how software is developed, maintained, and deployed. These incidents illustrate how a single weakness can cascade through supply chains, disrupt infrastructure, and erode trust in digital systems.\nModern development practices rely heavily on open-source components, with reuse accelerating exponentially. Between 2014 and 2023, software release frequency increased by 1466%, while known vulnerabilities (CVEs) grew by 463%3. The rise of supply chain attacks, including package poisoning and dependency confusion, further demonstrated how vulnerable the ecosystem has become.\nRegulatory Response: the Cyber Resilience Act #To address these risks, the EU introduced the Cyber Resilience Act (CRA). It establishes mandatory baseline security requirements for all products with digital elements marketed in the European Economic Area—including software, firmware, and hardware that connects to a network directly or indirectly during foreseeable use4.\nThe CRA applies across a wide landscape: from consumer IoT to embedded Linux systems, industrial controllers, and connected services. With over 40 billion connected devices projected by 20275, the regulation aims to raise the cybersecurity baseline for an increasingly interconnected world.\nSystems Impact #For developers and architects, the CRA codifies best practices as legal requirements: secure defaults, transparent supply chains, and timely vulnerability management. Meeting these obligations helps prevent large-scale exploits and simplifies assurance processes.\nFor customers and integrators, it brings visibility. The mandatory SBOM (Software Bill of Materials) enables faster risk assessment when CVEs are disclosed and makes product security postures auditable.\nSecurity is no longer assumed—it must be demonstrated. The CRA turns this into a shared, enforceable responsibility across the product lifecycle.\nWhat Software Providers and Developers Must Do #The Cyber Resilience Act introduces enforceable cybersecurity obligations for any company that develops, integrates, or distributes software in the EU market. Whether you\u0026rsquo;re building embedded systems, cloud-connected apps, or supplying third-party libraries, CRA compliance affects your release process, documentation, and lifecycle management.\nBuild Secure by Design # Eliminate known exploitable vulnerabilities before release Apply secure defaults (no default passwords, close unused ports) Minimize attack surface and enforce least privilege Implement secure data handling (encryption in transit and at rest) Integrate logging, monitoring, and mitigation mechanisms Use secure and vetted third-party components Document all of the above in your technical file Tip: Treat threat modeling and security reviews as release criteria—not optional QA steps.\nGenerate and Maintain an SBOM # Create a machine-readable Software Bill of Materials (SPDX, CycloneDX) Include all open-source and proprietary components Keep it current with every build and make it accessible to authorities on request Tip: Automate SBOM generation in your CI/CD pipeline.\nHandle Vulnerabilities Responsibly # Monitor CVEs relevant to your components Establish clear internal processes for triage, patching, and disclosure Support coordinated vulnerability disclosure (CVD) Tip: Maintain a public security contact and disclosure policy.\nReport Exploited Vulnerabilities Within 24 Hours # If your product is under active attack, you must report it to ENISA (the European Union Agency for Cybersecurity) 6 Have internal alerting and workflows ready to meet this deadline Tip: Link this process to your incident response runbook.\nDeliver Free Security Updates for At Least 5 Years # Declare your support period clearly Provide signed, secure updates for the full term Keep records of your delivery mechanisms and update history Tip: Include rollback or reset-to-safe mechanisms to meet “secure by default” expectations.\nComplete Technical Documentation (Annex VII) # Describe the product and software versions Include architecture, risk assessments, testing, update procedures, and SBOM This file must be ready before market placement and kept up to date Undergo the Right Conformity Assessment (Annex VIII) #Before a product can be placed on the EU market, manufacturers must demonstrate that it meets the CRA’s cybersecurity requirements. This is done through a conformity assessment—a formal process based on one of several predefined * modules*.\nModules are standardized procedures for proving compliance. They range from internal self-assessment to full third-party audits, depending on the product’s risk profile and classification.\nModule A – Internal Production Control Most software providers will use this self-assessment path. You document how the product complies with CRA requirements, prepare technical documentation (Annex VII), issue a Declaration of Conformity, and apply the CE marking—no external party involved.\nModules B + C – EU-Type Examination + Production Control For products classified as critical (Annex III), a notified body first reviews the product’s design (Module B). The manufacturer then ensures production matches that approved design (Module C).\nModule H – Full Quality Assurance For highly critical systems, this module requires a notified body to audit the manufacturer’s entire quality management system, including development, cybersecurity controls, and update procedures.\nTip: If your product functions as an OS, cryptographic module, network interface, or system-level component, consult Annex III early and prepare for third-party assessment.\nConclusion #The CRA sets a new standard for cybersecurity in digital products. It requires engineering teams to embed security into every phase—from design to updates. It gives customers better insight into product composition and risk.\nPreparing early ensures compliance and builds stronger systems. As attacks grow more sophisticated, the CRA turns resilience into a product requirement—not a feature.\nThese steps aren’t just checkboxes—they form a continuous process. The CRA makes it clear: cybersecurity is a development responsibility, a documentation responsibility, and a business responsibility.\nFurther Reading on the Cyber Resilience Act # PwC – Product Categories of the CRA\nEU CRA Portal\nBSI CRA Guidance – Secure by Design\nEuropean CRA Explainer\nEuropean CRA Central Resource\nReferences # Unveil the Mystery of Critical Software Vulnerabilities, ACM Queue, 2024.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nDecade Retrospective: The State of Vulnerabilities, Trustwave SpiderLabs Blog, 2024.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n10th Annual State of the Software Supply Chain, Sonatype, 2024.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nCRA Regulation on EUR-Lex (2024/2847).\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nState of IoT 2024, IoT Analytics.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nENISA (the European Union Agency for Cybersecurity)\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"May 19, 2025","permalink":"/posts/cyber-resilience-act/","section":"Posts","summary":"\u003cp\u003eOver the past decade\u003csup id=\"fnref:1\"\u003e\u003ca href=\"#fn:1\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e1\u003c/a\u003e\u003c/sup\u003e, numerous high-impact vulnerabilities, such as Heartbleed (2014), WannaCry (2017), SolarWinds (\n2020), and Log4Shell (2021)\u003csup id=\"fnref:2\"\u003e\u003ca href=\"#fn:2\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e2\u003c/a\u003e\u003c/sup\u003e. Have exposed systemic flaws in how software is developed, maintained, and deployed.\nThese incidents illustrate how a single weakness can cascade through supply chains, disrupt infrastructure, and erode\ntrust in digital systems.\u003c/p\u003e\n\u003cp\u003eModern development practices rely heavily on open-source components, with reuse accelerating exponentially. Between 2014\nand 2023, software release frequency increased by 1466%, while known vulnerabilities (CVEs) grew by 463%\u003csup id=\"fnref:3\"\u003e\u003ca href=\"#fn:3\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e3\u003c/a\u003e\u003c/sup\u003e. The rise\nof supply chain attacks, including package poisoning and dependency confusion, further demonstrated how vulnerable the\necosystem has become.\u003c/p\u003e","title":"Understanding The Cyber Resilience Act"},{"content":"","date":null,"permalink":"/tags/c++/","section":"Tags","summary":"","title":"C++"},{"content":"","date":null,"permalink":"/tags/debug-probe/","section":"Tags","summary":"","title":"Debug-Probe"},{"content":"Embedded systems development is constrained by limited resources, strict timing requirements, and close coupling with hardware. Real-time operating systems (RTOS) address these challenges through deterministic task scheduling and predictable interrupt handling. They ensure systems respond within defined time bounds even under load, shifting the focus from raw throughput to timing guarantees and reliability.\nIn the first post 1 we examined how real-time kernels such as Zephyr offer reusable APIs and robust middleware for safe, maintainable designs. The follow-up post 2 showed why Zephyr is a modern choice for MCU-based projects.\nThe goal of this post is to show how to build a minimal, C++-ready Zephyr project for the Raspberry Pi Pico using the freestanding application pattern. We will walk through toolchain setup, project scaffolding, environment configuration and flashing your first application—cleanly, repeatably and independently of the Zephyr source tree.\nWhat Is the Freestanding Application Pattern? #In Zephyr, the freestanding application pattern means your project lives in its own repository, separated from the Zephyr source tree. This keeps your codebase clean, simplifies version control, and lets you fully leverage Zephyr’s tooling without entangling your application with OS internals 3.\nSteps #This guide walks you through setting up a clean, C++-ready Zephyr project for the Raspberry Pi Pico. You’ll create an isolated workspace, install the essential SDK, configure your environment, and build and flash your application, while keeping your codebase independent of the Zephyr OS tree based on 4. The hardware setup, including how to wire up the Pico and connect the Debug Probe, follows the same approach described in 5.\nRequirements # Linux host (macOS and Windows also work) Raspberry Pi Pico (Model 1)6 Raspberry Pi Debug Probe7 Optional: LED wired as in the Embedded Rust with Pico guide5 Step 1 – Toolchain \u0026amp; Workspace Setup #This step creates an isolated build environment, checks out just the parts of Zephyr you need, and installs the Pico-compatible cross-compiler.\nSet up a Python virtual environment – keeps Zephyr tools from polluting the system Python.\npython3 -m venv ~/zephyr-venv source ~/zephyr-venv/bin/activate pip install west Initialize a workspace pinned to Zephyr v4.1.0 – west init adds a zephyrproject folder; --mr selects the release tag.\nwest init zephyrproject -m https://github.com/zephyrproject-rtos/zephyr --mr v4.1.0 cd zephyrproject Pull only required modules – west update --narrow skips optional drivers and HALs, saving bandwidth and disk space.\nwest update --narrow west zephyr-export west packages pip --install Install the Zephyr SDK toolchain – downloads just the arm-zephyr-eabi toolchain needed for the Raspberry Pi Pico.\nwest sdk install --toolchains arm-zephyr-eabi --version 0.17.0 Now, you have a clean workspace with the minimum tools required to build Zephyr firmware for the Pico.\nStep 2 – Verify Tool Versions #Modern Zephyr expects recent tooling; outdated versions are the cause of most early build errors.\nCheck the following versions\ncmake --version \u0026amp;\u0026amp; python3 --version \u0026amp;\u0026amp; dtc --version \u0026amp;\u0026amp; ninja --version \u0026amp;\u0026amp; west --version Compare against the minimums\nTool Role in build Min version CMake Generates Ninja build files 3.20 Python Runs West and helper scripts 3.10 DTC Compiles device-tree sources 1.6 Ninja Fast parallel build backend 1.10 West Zephyr’s meta-tool for fetch/build 1.0 If any tool is below these versions, upgrade it using your package manager or the official releases. For detailed instructions, refer to the official documentation3.\nStep 3 – Create a New Project #A template repo gives you a ready-made CMakeLists, sample prj.conf, and board setup for you focus on code, not scaffolding.\nClick Use this template ↗ and create your repo under GitHub.\nClone it locally and enter the directory:\ngit clone git@github.com:\u0026lt;your-username\u0026gt;/\u0026lt;your-new-project\u0026gt;.git cd \u0026lt;your-new-project\u0026gt; Edit CMakeLists.txt or prj.conf only as needed.\nStep 4 – Configure Environment Variables #An env.sh file pins all paths so every developer builds the same way.\nexport WORKSPACE=$HOME/zephyrproject # where Zephyr lives export BOARD=rpi_pico # default board export ZEPHYR_BASE=$WORKSPACE/zephyr # Zephyr source path export ZEPHYR_TOOLCHAIN_VARIANT=zephyr # use Zephyr SDK export ZEPHYR_SDK_INSTALL_DIR=$HOME/zephyr-sdk-0.17.0 Make the env.sh executable and source it before building:\nchmod +x env.sh source env.sh Step 5 – Build · Flash · Log #A basic Makefile is provided to simplify building and flashing without needing to switch directories or recall repetitive commands.\nBuild – compiles the app and generates zephyr.hex.\nmake build Flash – programs the Pico through the Debug Probe (SWD).\nmake flash Log – opens the UART console with Picocom.\nmake log Exit Picocom with Ctrl-A, then Ctrl-X.\nEach make target wraps the equivalent west build, west flash commands for consistency.\nC++ Integration Notes # Enable C++: add CONFIG_CPP=y and set the standard, e.g. CONFIG_STD_CPP14=y. Avoid static locals inside main(); they require guarded initialization. Use k_thread_create() or k_work for background work instead of std::thread. Summary #This guide helps you build a standalone Zephyr project for the Raspberry Pi Pico using a clean, template-based setup. It keeps your application independent of the Zephyr source tree, ensures reproducible builds with a shared env.sh file, and streamlines development using make targets for building, flashing, and logging. With minimal changes, you get a C++-ready environment tailored to the constraints of the Pico—making it fast to get started and easy to maintain.\nReferences # How an RTOS Simplifies Embedded Development – concept overview of RTOS benefits.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nZephyr RTOS: Introduction and Ecosystem Overview – survey of Zephyr’s architecture and community.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nZephyr Getting Started Guide – official steps for initializing and using Zephyr.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nZephyr Freestanding Application Pattern – explanation of the out-of-tree application model.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nEmbedded Rust with Pico – hands-on guide showing LED wiring and UART logging.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nRaspberry Pi Pico Documentation – overview and specifications.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nRaspberry Pi Debug Probe – SWD + UART adapter for flashing and logging.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"May 13, 2025","permalink":"/posts/zephyr-freestanding-cpp-rpi-pico/","section":"Posts","summary":"\u003cp\u003eEmbedded systems development is constrained by limited resources, strict timing requirements, and close coupling with\nhardware. Real-time operating systems (RTOS) address these challenges through deterministic task scheduling and\npredictable interrupt handling. They ensure systems respond within defined time bounds even under load, shifting the\nfocus from raw throughput to timing guarantees and reliability.\u003c/p\u003e\n\u003cp\u003eIn the first post \u003csup id=\"fnref:1\"\u003e\u003ca href=\"#fn:1\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e1\u003c/a\u003e\u003c/sup\u003e we examined how real-time kernels such as Zephyr offer reusable APIs and robust middleware for\nsafe, maintainable designs. The follow-up post \u003csup id=\"fnref:2\"\u003e\u003ca href=\"#fn:2\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e2\u003c/a\u003e\u003c/sup\u003e showed why Zephyr is a modern choice for MCU-based projects.\u003c/p\u003e","title":"Developing a Zephyr Freestanding C++ Application"},{"content":"","date":null,"permalink":"/categories/embedded/","section":"Categories","summary":"","title":"Embedded"},{"content":"","date":null,"permalink":"/tags/freestanding/","section":"Tags","summary":"","title":"Freestanding"},{"content":"","date":null,"permalink":"/tags/rpi_pico/","section":"Tags","summary":"","title":"Rpi_pico"},{"content":"","date":null,"permalink":"/categories/rtos/","section":"Categories","summary":"","title":"Rtos"},{"content":"","date":null,"permalink":"/categories/zephyr/","section":"Categories","summary":"","title":"Zephyr"},{"content":"","date":null,"permalink":"/tags/zephyr/","section":"Tags","summary":"","title":"Zephyr"},{"content":"","date":null,"permalink":"/tags/embedded/","section":"Tags","summary":"","title":"Embedded"},{"content":"","date":null,"permalink":"/categories/firmware/","section":"Categories","summary":"","title":"Firmware"},{"content":"","date":null,"permalink":"/tags/iot/","section":"Tags","summary":"","title":"IoT"},{"content":"","date":null,"permalink":"/categories/realtime/","section":"Categories","summary":"","title":"Real‑Time"},{"content":"","date":null,"permalink":"/tags/rtos/","section":"Tags","summary":"","title":"RTOS"},{"content":"Real‑time operating systems are increasingly important in IoT and embedded development. They offer the timing guarantees and responsiveness needed for reliable operation. Many modern projects benefit from this approach, as outlined in RTOS: Comfort vs Safety. The fundamentals of what an RTOS is and why it matters for developers are explained in more detail in How an RTOS Simplifies Embedded Development.\nBacked by the Linux Foundation, Zephyr targets embedded systems that demand both efficiency and flexibility. It is designed to run on a wide range of processor architectures1, with a modular structure that allows developers to include only what they need. Despite its compact footprint, it supports key capabilities such as networking and precise real‑time behavior2. The name “Zephyr a soft west wind” 3 captures its lightweight intent.\nThe project’s foundation comes from Virtuoso, a real‑time OS originally created by Eonic Systems in the 1990s for digital signal processors4. Wind River acquired Eonic in 2001 and eventually transformed Virtuoso into Wind River Rocket, released as a free RTOS in 20155. Just a year later, Wind River contributed Rocket’s kernel to the open‑source community, launching the Zephyr Project as a vendor‑neutral initiative under the Linux Foundation4.\nPhoto by Mila Young on Unsplash\nWhy Zephyr # Open Source \u0026amp; Permissive Licensing: Licensed under Apache 2.0, Zephyr can be used in commercial and personal projects without restrictive terms. Development is transparent, with open contribution and governance1.\nScalability \u0026amp; Compact Kernel: Runs on systems with only a few kilobytes of RAM4. Developers include only the components they need, optimizing the OS for sensors, wearables, and other minimalist devices6.\nBroad Architecture Support: Supports Arm, x86, RISC‑V, and other CPU families24. Hundreds of boards and SoCs already have upstream support, easing reuse across diverse projects1.\nFeature‑Rich for Embedded Use: Provides networking (IPv4/IPv6, Wi‑Fi, Bluetooth LE), file systems, drivers, sensor frameworks and OTA updates in the main distribution—reducing the need for external modules.\nSecurity and Reliability: Follows a strict secure‑development lifecycle and aims for compliance with standards such as functional safety and PSA2. Options like memory protection and thread isolation make it suitable for industrial use6.\nEcosystem #Zephyr is more than a kernel—it is a full platform backed by a growing ecosystem. Vendor‑neutral governance1 attracts major contributors, including Intel, NXP, Nordic Semiconductor, and Texas Instruments4.\nA standout strength is hardware coverage: more than 170 boards are supported, giving Zephyr one of the broadest hardware footprints among RTOSes1. Developers can often run it on off‑the‑shelf hardware or extend support using existing frameworks6.\nThe toolchain uses familiar tools like CMake and Kconfig for configuration, and the west meta‑tool for managing multiple repositories—an approach that benefits developers with a software engineering background. While powerful for managing complex codebases, this setup may be less intuitive for teams coming from traditional hardware-centric workflows.\nZephyr maintains a high release cadence, with long-term support versions and rigorous QA4. Community contributions drive steady improvements, with features, drivers, and security updates frequently merged into mainline. Commercial variants such as Rocket, backed by firms like Wind River, add options like cloud integration and extended support4.\nRTOS Landscape #Zephyr is one of many RTOS options. Open-source FreeRTOS provides a minimal baseline ideal for ultra-constrained designs, while commercial systems like VxWorks7 and SafeRTOS8 target safety-critical use cases. Benchmarks from UL Solutions9 and 10 highlight strengths across different systems—Zephyr leads in some inter-thread communication tests, while FreeRTOS excels in task-switch latency.\nWhile Zephyr offers more features out of the box, this can lead to added complexity and memory overhead compared to leaner kernels6. Developers often note a steeper learning curve and evolving tooling, which can make the transition from simpler stacks more difficult6. Despite this, Zephyr’s modular design, long-term roadmap, and neutral governance make it a strong candidate for a wide range of embedded applications.\nSummary #Zephyr offers a compact yet versatile RTOS for connected devices. With permissive licensing, a robust feature set, and broad industry backing, it provides a solid foundation for building secure, scalable embedded applications when its capabilities match project needs.\nReferences # About the Zephyr Project\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nWhat Is Zephyr?\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nZephyr\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nZephyr (operating system)\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nWind River Sets Rocket RTOS On Free Trajectory\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nZephyr RTOS Review: The Best Open Source OS for IoT?\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nVxWorks: Real-Time Operating System for the Intelligent Edge \u0026#160;\u0026#x21a9;\u0026#xfe0e;\nPre-Certified Safety RTOS\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nMeasuring Real-Time Operating System Performance – Part I: Finding and Porting a Benchmark Suite\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nMeasuring Real-Time Operating System Performance – Part II: Comparing FreeRTOS vs. Zephyr\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"May 7, 2025","permalink":"/posts/zephyr-intro/","section":"Posts","summary":"\u003cp\u003eReal‑time operating systems are increasingly important in IoT and embedded development. They offer the timing guarantees\nand responsiveness needed for reliable operation. Many modern projects benefit from this approach, as outlined\nin \u003ca href=\"https://advt3.com/posts/real_time_os_introduction/\" target=\"_blank\" rel=\"noreferrer\"\u003eRTOS: Comfort vs Safety\u003c/a\u003e. The fundamentals of what an RTOS is\nand why it matters for developers are explained in more detail\nin \u003ca href=\"https://advt3.com/posts/rtos-development/\" target=\"_blank\" rel=\"noreferrer\"\u003eHow an RTOS Simplifies Embedded Development\u003c/a\u003e.\u003c/p\u003e\n\u003cp\u003eBacked by the Linux Foundation, Zephyr targets embedded systems that demand both efficiency and flexibility. It is\ndesigned to run on a wide range of processor architectures\u003csup id=\"fnref:1\"\u003e\u003ca href=\"#fn:1\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e1\u003c/a\u003e\u003c/sup\u003e, with a modular structure that allows developers to\ninclude only what they need. Despite its compact footprint, it supports key capabilities such as networking and precise\nreal‑time behavior\u003csup id=\"fnref:2\"\u003e\u003ca href=\"#fn:2\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e2\u003c/a\u003e\u003c/sup\u003e. The name “Zephyr a soft west wind” \u003csup id=\"fnref:3\"\u003e\u003ca href=\"#fn:3\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e3\u003c/a\u003e\u003c/sup\u003e captures its lightweight intent.\u003c/p\u003e","title":"Zephyr RTOS: Introduction and Ecosystem Overview"},{"content":"Systems that must guarantee safety or deliver peak performance rely on tasks finishing within strict, predictable time limits1. Missing even a single deadline can cascade into system-wide failures with serious consequences.\nThese timing requirements extend far beyond aerospace and manufacturing. Today, robotics, connected vehicles, wearables, and smart infrastructure all depend on coordinated timing, power management, and connectivity. In such environments, a Real-Time Operating System (RTOS) can be a game-changer.\nAn RTOS offers more than basic task scheduling. It establishes a uniform runtime environment with deterministic behavior, organized concurrency, and dependable interfaces for interrupts, communications, and power control—building blocks for robust, testable, and maintainable software2.\nMarket analysts expect RTOS adoption to surge over the next decade, with the sector projected to exceed USD 12 billion by 2034, driven by automation, edge intelligence, and embedded AI trends3.\nIn this article, we’ll explore how an RTOS enhances your development workflow and outline the factors teams should weigh before integrating one into their systems.\nPhoto by Rodion Kutsaiev on Unsplash\nWhy Use an RTOS? #As embedded projects grow in scope and ambition, ad-hoc code often hits its limits. An RTOS brings a proven framework that accelerates delivery and ensures quality:\nGuaranteed timing Built-in schedulers and hardware timers enforce deadlines for critical tasks in control loops, signal processing, and real-time monitoring.\nConcurrent task handling Preemptive, priority-driven scheduling lets multiple tasks run safely in parallel, replacing error-prone state machines with clean abstractions.\nHardware independence Standardized APIs for peripherals (GPIO, I²C, SPI, UART, timers) minimize platform-specific code and simplify porting between MCUs.\nReusable middleware Shared components—network stacks, filesystems, device drivers—are available out of the box, reducing custom code and fostering consistency.\nBuilt-in testability Clear task boundaries and deterministic execution models support unit and integration testing, making it easier to validate behavior and debug issues.\nScalable architecture From minimal MCUs to complex systems with storage and connectivity, an RTOS framework adapts without forcing major rewrites.\nEasier maintenance Modular drivers and layered services let teams update or replace components with confidence over the product lifecycle.\nSafety and certification Many RTOSes include features—task isolation, trace logs, documentation tooling—geared toward ISO 26262, IEC 61508, and DO-178C compliance.\nBy offloading common infrastructure concerns to the RTOS, engineers can focus their efforts on the unique value of their application.\nWhat an RTOS Provides #At its core, an RTOS supplies the following essential services:\nDeterministic scheduler A preemptive, priority-based engine that ensures critical tasks run at the right time, every time.\nInter-task synchronization Queues, semaphores, and mutexes for safe data sharing and task coordination without resorting to custom locking schemes.\nPredictable interrupt handling Integrated interrupt dispatch with bounded latency, so external events feed into tasks reliably.\nPeripheral abstraction Consistent interfaces for hardware access—timers, GPIO, DMA, communication buses—across different platforms.\nEnergy management Native support for sleep modes and dynamic frequency scaling to maximize battery life in resource-constrained designs.\nThese capabilities form the bedrock of any real-time application, ensuring systems remain responsive and adaptable as requirements evolve.\nFor deeper detail on these features, see 4 and 5.\nPlatform Features That Save Time #Beyond the kernel, mature RTOS distributions include tools and services to speed development:\nBoard Support Packages (BSPs) Ready-to-use configurations for popular microcontrollers, handling pin mapping and peripheral setup without manual intervention.\nIntegrated protocol stacks Built-in support for MQTT (messaging), CoAP (constrained REST), TLS (secure transport), BLE (wireless), and more—no custom stack required.\nTesting and simulation Frameworks for unit and integration tests plus emulators like QEMU let teams validate firmware long before hardware arrives.\nComprehensive toolchains Bundled compilers, debuggers, runtime trace, and CI integrations catch timing issues and logic errors early in the pipeline.\nThese extras cut down on boilerplate work and let teams iterate rapidly, from first prototype to production release6.\nKey Trade-offs and Constraints #While an RTOS can streamline development and boost reliability, it also brings new considerations. The list below highlights critical factors—though it isn’t exhaustive:\nMemory and CPU overhead The kernel, middleware, and context switches consume RAM, flash, and processing cycles you must budget for.\nLatency budgeting Scheduler ticks and system calls introduce delays; include them in worst-case execution time analyses.\nDebugging complexity Multithreaded designs can suffer race conditions, deadlocks, and priority inversion. Invest in trace tools and disciplined test practices.\nSecurity and safety Examine support for MPU/MMU separation, secure boot, and reliable over-the-air update mechanisms.\nCertification requirements Achieving ISO 26262, IEC 61508, or DO-178C often demands vendor documentation, trace logs, and formal process evidence.\nLicensing and ecosystem Some RTOSes require commercial licenses or proprietary tools; weigh cost, support, and community activity.\nToolchain alignment Ensure the RTOS SDK integrates smoothly with your build system, debugger, CI pipeline, and test frameworks.\nTeam expertise Hard real-time development and safety-critical processes call for specialized skills—plan training or hiring accordingly.\nVendor lock-in Proprietary APIs and closed-source components can limit future migration or ecosystem choices.\nSee 7 for a concise checklist of these considerations; for deeper dives, consult 8 and 9.\nConclusion #An RTOS does more than guarantee timing—it provides a structured foundation for building complex, reliable embedded systems. By centralizing task management, hardware abstraction, and middleware, an RTOS frees teams to focus on their product’s unique features.\nWhen real-time performance, safety requirements, or modularity become non-negotiable, adopting an RTOS early can reduce risk, accelerate delivery, and create a scalable, maintainable architecture.\nReferences # https://www.ibm.com/think/topics/real-time-operating-system\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nhttps://www.ni.com/en/shop/data-acquisition-and-control/add-ons-for-data-acquisition-and-control/what-is-labview-real-time-module/what-is-a-real-time-operating-system--rtos--.html\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nhttps://www.marketresearchfuture.com/reports/real-time-operating-system-market-34731\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nhttps://www.suse.com/c/what-is-a-real-time-operating-system/\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nhttps://blackberry.qnx.com/en/ultimate-guides/what-is-real-time-operating-system\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nhttps://www.freertos.org/Why-FreeRTOS/FAQs/What-is-this-all-about/\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nhttps://sternumiot.com/iot-blog/crash-course-introduction-to-real-time-operating-system-rtos/\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nhttps://www.windriver.com/solutions/learning/rtos\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nhttps://www.aptiv.com/en/insights/article/what-is-a-real-time-operating-system\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"May 1, 2025","permalink":"/posts/rtos-development/","section":"Posts","summary":"\u003cp\u003eSystems that must guarantee safety or deliver peak performance rely on tasks finishing within strict, predictable time limits\u003csup id=\"fnref:1\"\u003e\u003ca href=\"#fn:1\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e1\u003c/a\u003e\u003c/sup\u003e. Missing even a single deadline can cascade into system-wide failures with serious consequences.\u003c/p\u003e\n\u003cp\u003eThese timing requirements extend far beyond aerospace and manufacturing. Today, robotics, connected vehicles, wearables, and smart infrastructure all depend on coordinated timing, power management, and connectivity. In such environments, a Real-Time Operating System (RTOS) can be a game-changer.\u003c/p\u003e\n\u003cp\u003eAn RTOS offers more than basic task scheduling. It establishes a uniform runtime environment with deterministic behavior, organized concurrency, and dependable interfaces for interrupts, communications, and power control—building blocks for robust, testable, and maintainable software\u003csup id=\"fnref:2\"\u003e\u003ca href=\"#fn:2\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e2\u003c/a\u003e\u003c/sup\u003e.\u003c/p\u003e","title":"How an RTOS Simplifies Embedded Development"},{"content":"Safety and performance-critical systems rely on tasks consistently meeting their deadlines1. If even one deadline is missed, the system may fail—with potentially catastrophic results.\nThese timing constraints are no longer limited to aerospace or factory lines. They now define the reliability of robotics, industrial automation, connected vehicles, and even wearable devices. If your system must coordinate timing, power, and connectivity, a Real‑Time Operating System (RTOS) may be essential.\nAn RTOS is more than a scheduler. It provides deterministic execution, structured concurrency, and standardized interfaces for interrupts, communication, and power management—core foundations for building correct, testable, and maintainable systems2.\nRTOS adoption is projected to nearly double by 2034, reaching over USD 12 billion, driven by automation, edge computing, and embedded AI workloads3.\nSo what does an RTOS really add to your development workflow—and what should you consider before adopting one? Read on.\nPhoto by Rodion Kutsaiev on Unsplash\nWhy Use an RTOS? #As embedded systems evolve, so do expectations for reliability, modularity, and faster development cycles.\nBare-metal code can meet strict timing constraints, but it often struggles with scaling, reusability, and long-term maintainability.\nRTOS provides the structure to build modern embedded applications that are robust, portable, and production-ready—without sacrificing low-level control.\nPrecise timing\nBuilt-in schedulers and timers ensure that critical tasks execute within defined deadlines, essential for control systems, signal processing, and real-time monitoring.\nTask concurrency\nAn RTOS manages multiple independent tasks with preemptive, priority-based scheduling. This eliminates complex state machines and simplifies control logic.\nHardware abstraction\nStandard APIs for peripherals (GPIO, I2C, SPI, UART, timers) make applications more portable and reduce time spent adapting to new platforms.\nCode reusability\nMiddleware components—such as network stacks, filesystems, and device drivers—can be reused across projects, improving consistency and reducing development effort.\nTestability\nRTOS-based systems support unit and integration testing with clearer task boundaries and deterministic behavior, making them easier to validate and debug.\nScalability\nRTOS architectures can grow from minimal deployments on small MCUs to full-featured systems with connectivity, storage, and dynamic resource management.\nMaintainability\nClear task separation, modular drivers, and layered middleware lead to systems that are easier to refactor and update over time.\nCompliance and certification\nMany RTOS platforms are designed with safety standards in mind, offering task isolation, traceability, and documentation support for certifications like ISO 26262, IEC 61508, or DO-178C.\nBy introducing these capabilities, an RTOS helps teams focus on building the functionality that matters—without rebuilding the same low-level infrastructure for every new project.\nWhat an RTOS Provides #An RTOS provides consistent building blocks for developing responsive, reliable, and maintainable embedded systems. These cores feature abstract away timing-critical control logic and enable structured system design:\nDeterministic scheduling\nA preemptive, priority-based scheduler ensures tasks run when they should. This predictable timing model is key to meeting hard real-time constraints and avoiding jitter in time-sensitive applications.\nSafe inter-task communication\nBuilt-in primitives like queues, semaphores, and mutexes help tasks coordinate and share data safely. These eliminate ad-hoc synchronization code and reduce the risk of race conditions or deadlocks.\nBounded interrupt response\nInterrupt handling integrates tightly with the scheduler, offering consistent latency. Developers can prioritize responses while still maintaining system stability.\nHardware abstraction\nRTOS APIs wrap low-level peripheral access (timers, GPIO, buses, DMA) in consistent interfaces. This reduces platform-specific code and improves portability across MCU families.\nPower management\nBuilt-in support for sleep states and dynamic frequency scaling helps optimize power use, which is critical for battery-powered and energy-constrained devices.\nThese features are foundational—not just for timing—but for building embedded systems that scale and survive across hardware revisions and feature growth. For additional details on these RTOS features, see 4 and 5.\nPlatform Features That Save Time #Modern RTOS platforms don’t stop at task control—they come bundled with tooling and middleware that accelerate bring-up and reduce time-to-market6:\nBoard support packages (BSPs)\nPredefined configurations for popular MCUs simplify hardware initialization, pin mapping, and peripheral setup, allowing teams to start development without low-level customization.\nBuilt-in protocol stacks\nNative support for communication protocols like MQTT (messaging), CoAP (constrained REST), TLS (secure transport), and BLE (short-range wireless) allows teams to build connected products without implementing these stacks from scratch.\nTesting and simulation support\nMany platforms include unit testing frameworks, integration test hooks, and emulation environments like QEMU. These allow teams to validate code early—even before hardware is available—and support automated regression testing.\nIntegrated tooling\nRTOS SDKs typically include toolchains with compiler support, debuggers, runtime trace, and CI-friendly build systems. These tools help detect timing issues, resource contention, and logic errors during development.\nTogether, these platform-level capabilities help teams avoid common setup work, shorten the path to working firmware, and create a development flow that is easier to test, reuse, and scale.\nKey Trade‑offs and Constraints #While RTOS can streamline development and improve reliability, it also introduces constraints that shape your design, tooling, and team requirements. The following highlights key factors to consider before making a decision:\nMemory and CPU footprint\nKernel, protocol stacks, and drivers consume RAM and flash; context‑switch and interrupt overhead must still fit worst‑case deadlines.\nLatency overhead\nScheduler ticks, system calls, and interrupt masking add microseconds that must be budgeted into timing analysis.\nDebugging complexity\nConcurrency introduces race conditions, deadlocks, and priority inversion. Plan for trace tools, priority‑aware unit tests, and disciplined code reviews.\nSecurity and safety model\nConfirm MPU/MMU support, privilege separation, secure boot, and robust update mechanisms.\nCertification effort\nISO 26262, IEC 61508, or DO‑178C compliance may require vendor documentation, audit artifacts, and long‑term support contracts.\nLicensing and ecosystem maturity\nWeigh commercial terms, source availability, community activity, and third‑party middleware support.\nToolchain and workflow integration\nVerify that the RTOS SDK aligns with your build system, debugger, CI pipeline, and test frameworks.\nDeveloper expertise and hiring\nRTOS design, hard‑real‑time debugging, and safety processes demand specialised skills. Budget for training or factor in the scarcity of experienced engineers.\nVendor lock‑in risk\nProprietary APIs or closed drivers can hinder future platform migration and limit ecosystem choice.\nSee 7 for a concise checklist of these considerations; 8 and 9 provide deeper analysis.\nConclusion #RTOS platforms offer more than just deterministic scheduling—they bring structure, reusability, and development efficiency to modern embedded systems. They help teams manage growing complexity, meet strict timing constraints, and build on proven, portable components.\nWhile not always required, an RTOS becomes essential when systems demand real-time behavior, safety guarantees, or modular design. Choosing the right platform and evaluating its trade-offs early can reduce development risk, accelerate time to market, and lay the groundwork for a scalable and maintainable system.\nReferences # https://www.ibm.com/think/topics/real-time-operating-system\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nhttps://www.ni.com/en/shop/data-acquisition-and-control/add-ons-for-data-acquisition-and-control/what-is-labview-real-time-module/what-is-a-real-time-operating-system--rtos--.html\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nhttps://www.marketresearchfuture.com/reports/real-time-operating-system-market-34731\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nhttps://www.suse.com/c/what-is-a-real-time-operating-system/\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nhttps://blackberry.qnx.com/en/ultimate-guides/what-is-real-time-operating-system\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nhttps://www.freertos.org/Why-FreeRTOS/FAQs/What-is-this-all-about/\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nhttps://sternumiot.com/iot-blog/crash-course-introduction-to-real-time-operating-system-rtos/\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nhttps://www.windriver.com/solutions/learning/rtos\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nhttps://www.aptiv.com/en/insights/article/what-is-a-real-time-operating-system\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"May 1, 2025","permalink":"/posts/rtos-embedded-development/","section":"Posts","summary":"\u003cp\u003eSafety and performance-critical systems rely on tasks consistently meeting their deadlines\u003csup id=\"fnref:1\"\u003e\u003ca href=\"#fn:1\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e1\u003c/a\u003e\u003c/sup\u003e. If even one deadline is missed, the system may fail—with potentially catastrophic results.\u003c/p\u003e\n\u003cp\u003eThese timing constraints are no longer limited to aerospace or factory lines. They now define the reliability of robotics, industrial automation, connected vehicles, and even wearable devices. If your system must coordinate timing, power, and connectivity, a Real‑Time Operating System (RTOS) may be essential.\u003c/p\u003e\n\u003cp\u003eAn RTOS is more than a scheduler. It provides deterministic execution, structured concurrency, and standardized interfaces for interrupts, communication, and power management—core foundations for building correct, testable, and maintainable systems\u003csup id=\"fnref:2\"\u003e\u003ca href=\"#fn:2\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e2\u003c/a\u003e\u003c/sup\u003e.\u003c/p\u003e","title":"How an RTOS Simplifies Embedded Development"},{"content":"Overview #Air-quality monitoring is everywhere—from smart-home gadgets to safety-critical industrial gear.12. The sensors may look alike, yet the software behind them serves very different missions.\nTwo Scenarios, One Question #Living-room comfort\nYou’re sitting on the sofa in your living room while a smart sensor quietly samples carbon dioxide and volatile organic compounds (VOCs) every few minutes. It logs the data, draws trend lines in your app, and may open a window or start a fan. A brief delay or missed sample hardly matters—indoor air changes slowly.\nEngine-room safety\nNow step into the engine room of an oil-cargo ship. Here, the monitor scans for carbon monoxide and hydrogen sulfide—gases lethal at trace levels and common in machinery spaces.34\nIf either spikes, the system must alarm, start forced ventilation, and time-stamp the event within milliseconds. At high concentrations those gases can knock a person unconscious in under 30 seconds, with death following soon after.\nSame sensing goals, opposite stakes, so the firmware cannot be the same.\nImage by Jungsik Kwak via Pixabay\nWhen Timing Becomes the Hazard #In a living room, a two-second delay is harmless, so a straightforward super-loop or cooperative scheduler suffices.5\nIn an engine room, lateness becomes a hazard. Guaranteeing deadlines under load requires a Real-Time Operating System (RTOS).6\nWhat an RTOS Provides # Deterministic, pre-emptive scheduling – The gas-alarm task seizes the CPU the instant a threshold trips. Bounded interrupt latency – The worst-case delay from sensor edge to handler is known and testable. Priority inheritance – A low-priority logger can’t block the alarm siren. Graceful overload handling – Non-critical chores are shed so life-saving code still meets its deadline. Why It Matters #For comfort devices, timing is flexible; a late sample only postpones a breeze. For safety gear, timing is the requirement. Detecting H₂S even 500 ms too late can mean dizziness, collapse, or worse.4\nConclusion #Air-quality monitors may share hardware, but their software foundations diverge sharply. When milliseconds mark the line between comfort and catastrophe, an RTOS isn’t a luxury add-on, it’s the bedrock.\nReferences # United Nations Environment Program, “How is air quality measured?” 2022\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nAirly, “How to measure air pollution?” Blog post, 2024\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nH. K. Nguyen et al., “Impacts of fuel quality on indoor environment onboard a ship,” Transportation Research Part D: Transport and Environment, vol. 86, 2020\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nOccupational Safety and Health Administration, “Hydrogen Sulfide — Standards,” 29 CFR 1910.1000 Table Z-2\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n“Building DC Energy Systems: RTOS vs. Super Loop”\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nWind River, “What Is a Real-Time Operating System (RTOS) and How Does It Work?” Learning Center\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"April 25, 2025","permalink":"/posts/real_time_os_introduction/","section":"Posts","summary":"\u003ch2 id=\"overview\" class=\"relative group\"\u003eOverview \u003cspan class=\"absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100\"\u003e\u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700\" style=\"text-decoration-line: none !important;\" href=\"#overview\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\u003c/span\u003e\u003c/h2\u003e\u003cp\u003eAir-quality monitoring is everywhere—from smart-home gadgets to safety-critical industrial gear.\u003csup id=\"fnref:1\"\u003e\u003ca href=\"#fn:1\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e1\u003c/a\u003e\u003c/sup\u003e\u003csup id=\"fnref:2\"\u003e\u003ca href=\"#fn:2\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e2\u003c/a\u003e\u003c/sup\u003e.\nThe sensors may look alike, yet the software behind them serves very different missions.\u003c/p\u003e\n\u003ch2 id=\"two-scenarios-one-question\" class=\"relative group\"\u003eTwo Scenarios, One Question \u003cspan class=\"absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100\"\u003e\u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700\" style=\"text-decoration-line: none !important;\" href=\"#two-scenarios-one-question\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\u003c/span\u003e\u003c/h2\u003e\u003cp\u003e\u003cstrong\u003eLiving-room comfort\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eYou’re sitting on the sofa in your living room while a smart sensor quietly samples carbon dioxide and volatile organic compounds (VOCs) every few minutes. It logs the data, draws trend lines in your app, and may open a window or start a fan. A brief delay or missed sample hardly matters—indoor air changes slowly.\u003c/p\u003e","title":"RTOS: Comfort vs Safety"},{"content":"Introduction and Overview #Industrial IoT (IIoT) is reshaping manufacturing by moving intelligence closer to on-site machinery and sensors. This edge-centric approach helps address challenges such as high latency, network bandwidth constraints, and unplanned downtime—which can exceed $260,000 per hour1. Single Board Computers (SBCs) have become a key element in modern Industrial PCs (IPCs) by offering the reliability and performance needed to reduce downtime and potentially improve ROI12. For further context on the evolution and industrial application of SBCs, overviews are available from LattePanda3 and Maxtang4.\nIn this article, we explore the role of SBCs at the industrial edge. The discussion covers practical aspects such as cost efficiency, predictive maintenance, and cybersecurity enhancements. By examining how SBC-driven IPCs contribute to resilient and cost-effective manufacturing environments, this article aims to provide valuable insights for those interested in understanding the evolving landscape of Industry 4.0.\nEdge Intelligence and Predictive Maintenance #Processing data locally enables predictive maintenance systems to detect anomalies early. For example, an SBC-based system that monitors vibration and temperature can alert engineers to early signs of wear, preventing costly breakdowns56. Additionally, high-performance modules like NVIDIA Jetson extend these capabilities to edge AI tasks—including autonomous robotics and real-time quality inspection—while ensuring data privacy and prompt responses in safety-critical environments789.\nWhy SBCs Are Integral to Modern IPCs #SBC-based hardware integrates processor, memory, storage, and I/O onto a single circuit board, offering several advantages:\nLower Costs:\nConsolidating components reduces material and assembly expenses, which helps control capital expenditures (CapEx) during large-scale deployments. This streamlined design lowers the total cost of ownership (TCO)42.\nFewer Moving Parts:\nWith fanless heat sinks and solid-state storage replacing mechanical components, SBCs minimize hardware failures.\nReliability in Extreme Conditions:\nIndustrial SBCs are built to operate in harsh environments with wide temperature ranges and robust enclosures. Secure attachment of components enhances shock and vibration resistance, ensuring dependable performance10.\nSecurity and Remote Management:\nModern IPCs based on SBCs include features such as secure boot, Trusted Platform Module (TPM) options, and remote firmware updates. These features help safeguard against cyberattacks and simplify maintenance211.\nReal-World Impact: Cost Savings and ROI #The benefits of SBC-based IPCs translate directly into financial gains:\nReduced CapEx:\nThe consolidated hardware design means fewer components are needed, lowering upfront costs4.\nLower OpEx:\nEnergy-efficient designs, fewer mechanical parts, and remote management contribute to reduced operational expenses2.\nMinimal Downtime:\nEffective predictive maintenance combined with robust cybersecurity measures minimizes costly disruptions, protecting revenue streams5612.\nAs edge computing evolves, the value of SBC-based IPCs will grow, delivering a blend of performance, energy efficiency, and security that drives ROI for manufacturers worldwide.\nAddressing Maintainability and Cybersecurity #To fully realize the benefits of SBC-based systems, robust lifecycle management is essential:\nRegular Updates:\nRemotely deploying firmware, OS, and application patches via over-the-air (OTA) updates keeps large fleets secure and operational2.\nSecure Boot and TPM:\nThese embedded security features verify firmware integrity and prevent unauthorized modifications, forming a critical defense against cyberthreats211.\nNetwork Segmentation:\nIsolating sensitive segments of the production network ensures that any potential compromise remains contained.\nThese strategies work together to maintain a resilient edge infrastructure and safeguard ROI.\nConclusion #Single Board Computers have become foundational to modern Industrial PCs by merging compact hardware design with industrial-grade reliability, security, and manageability. Embracing SBC-based architectures at the edge allows organizations to reduce costs, streamline maintenance, and fortify operations against disruptions. Supported by industry insights138, this robust integration underscores why SBC-driven IPCs are central to the evolution of Industry 4.0.\nReferences # IIoT World – The Actual Cost of Downtime\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nThings Embedded – Choosing an Embedded Motherboard or Single Board Computer\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nLattePanda – Definition, Evolution and Application of SBC\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nMaxtang – Understanding Single Board Computers\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nIndustrial Edge Computing and Field Interaction – ADVT3\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nEdge Computing Challenges – ADVT3\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nAdvent of Edge Computing – ADVT3\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nLattePanda – SBC Edge AI Use Cases\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nScienceDirect – AI and IIoT Integration\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nVersaLogic – Myth of \u0026ldquo;Extended Temperature\u0026rdquo; Embedded Computers\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nCoventry University – SBCs as Smart Sensors\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nWevolver – Exploring Edge AI with SBCs\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"March 23, 2025","permalink":"/posts/single-board-computer/","section":"Posts","summary":"\u003ch2 id=\"introduction-and-overview\" class=\"relative group\"\u003eIntroduction and Overview \u003cspan class=\"absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100\"\u003e\u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700\" style=\"text-decoration-line: none !important;\" href=\"#introduction-and-overview\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\u003c/span\u003e\u003c/h2\u003e\u003cp\u003eIndustrial IoT (IIoT) is reshaping manufacturing by moving intelligence closer to on-site machinery and sensors. This\n\u003cem\u003eedge-centric\u003c/em\u003e approach helps address challenges such as high latency, network bandwidth constraints, and \u003cstrong\u003eunplanned\ndowntime\u003c/strong\u003e—which can exceed $260,000 per hour\u003csup id=\"fnref:1\"\u003e\u003ca href=\"#fn:1\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e1\u003c/a\u003e\u003c/sup\u003e. Single Board Computers (SBCs) have become a key element in modern\nIndustrial PCs (IPCs) by offering the reliability and performance needed to reduce downtime and potentially improve\nROI\u003csup id=\"fnref1:1\"\u003e\u003ca href=\"#fn:1\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e1\u003c/a\u003e\u003c/sup\u003e\u003csup id=\"fnref:2\"\u003e\u003ca href=\"#fn:2\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e2\u003c/a\u003e\u003c/sup\u003e. For further context on the evolution and industrial application of SBCs, overviews are available from\nLattePanda\u003csup id=\"fnref:3\"\u003e\u003ca href=\"#fn:3\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e3\u003c/a\u003e\u003c/sup\u003e and Maxtang\u003csup id=\"fnref:4\"\u003e\u003ca href=\"#fn:4\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e4\u003c/a\u003e\u003c/sup\u003e.\u003c/p\u003e","title":"The Importance of Single Board Computers for the Industrial Edge"},{"content":"Industrial automation is evolving rapidly, driven by the need for higher efficiency, flexibility, and safety in modern manufacturing. As production lines become connected and responsive, manufacturers use data-driven methods to optimize operations, cut downtime, and adapt faster to market demands. However, these gains come with challenges: handling large device fleets, meeting strict safety and security standards, and integrating older systems with modern technologies. Real-world insights further highlight the importance of strong edge solutions, as shown in 1 and 2.\nGenerated by StockCake.com\nEmbedded Linux systems are central to this transformation. They offer versatility, solid security, and easy integration, helping industrial automation platforms reach their potential. By powering high-performance Industrial Computers (IPCs) and securing on-site processing and real-time data insights, these systems meet crucial needs across the manufacturing sector. As noted in 3, using Linux frameworks can boost both data processing and operational efficiency.\nAdvancing Local Processing and Predictive Analytics #Flexible operating systems like Linux enable efficient data analysis at production sites, reducing delays and allowing faster responses to real-world demands. This is key for predictive maintenance and quality control, where local processing can spot anomalies and streamline workflows. According to 4, these solutions will see strong growth, with global spending on localized computing expected to reach $378 billion by 2028 5. Their scalability ensures even complex manufacturing setups run efficiently.\nTailored Solutions and Legacy System Integration #Industrial environments often require solutions that combine older hardware with emerging technologies. Linux, paired with tools like the Yocto Project 6, lets developers build custom operating systems for tasks like robotics and process optimization. As shown in 7, industries gain from tailored configurations that integrate smoothly with existing systems. This extends the life of older equipment, reduces disruptions, and enables efficient processes 8.\nSecurity, Scalability, and Operational Control #As the industrial sector becomes interconnected, security and centralized management are crucial. Linux systems offer features such as Secure Boot and SELinux, helping prevent unauthorized access and protect vital operations. They also support the creation of Software Bills of Materials (SBOMs) to comply with standards like IEC 62443. Cloud-enabled management platforms further aid in configuring large fleets 9. These security and management tools align with key trends highlighted in 10.\nConclusion #Modern Linux systems are reshaping industrial automation by enabling real-time data analysis, strengthening security, and uniting older equipment with new technologies. These systems help manufacturers move toward efficient, reliable, and adaptive processes. As industrial networks become complex, the ability to securely manage large device fleets and meet strict standards remains crucial. Linux-based solutions address today’s problems and pave the way for sustainable, innovative, and competitive manufacturing. Their flexibility and power make them essential to the future of smart factories and adaptive production.\nReferences # Industrial Edge Computing: Field Interaction\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nEdge Computing Challenges\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nAdvent Edge Computing\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n2025 IT Infrastructure Trends: The Edge Computing, HCI, And AI Boom\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nGlobal Spending on Edge Computing to Reach $378 Billion by 2028\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nYocto Project Documentation\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nIndustrial and Heavy Equipment Case Studies\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nEdge Computing: The Key to Smarter Industrial Automation\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nWhy Use Linux-Based Embedded Devices in Industrial Applications?\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nTop Innovative Technology Trends for Manufacturing Operations in 2024\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"December 21, 2024","permalink":"/posts/driving_industrial_automation/","section":"Posts","summary":"\u003cp\u003eIndustrial automation is evolving rapidly, driven by the need for higher efficiency, flexibility, and safety in modern\nmanufacturing. As production lines become connected and responsive, manufacturers use data-driven methods to\noptimize operations, cut downtime, and adapt faster to market demands. However, these gains come with challenges:\nhandling large device fleets, meeting strict safety and security standards, and integrating older systems with modern\ntechnologies. Real-world insights further highlight the importance of strong edge solutions, as shown in \u003csup id=\"fnref:1\"\u003e\u003ca href=\"#fn:1\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e1\u003c/a\u003e\u003c/sup\u003e and \u003csup id=\"fnref:2\"\u003e\u003ca href=\"#fn:2\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e2\u003c/a\u003e\u003c/sup\u003e.\u003c/p\u003e","title":"Driving Efficiency and Security in Industrial Automation with Embedded Linux"},{"content":"","date":null,"permalink":"/tags/cross-compilation/","section":"Tags","summary":"","title":"Cross-Compilation"},{"content":"In embedded systems and industrial edge computing, using a reliable, efficient, and safe language like Rust is crucial for developing applications where performance and memory safety are critical. This post presents how to develop Rust-based applications using Yocto Linux\u0026rsquo;s default Rust compiler setup without relying on the meta-rust layer.\nImage by Saad Salim from unsplash\nThis guide focuses on configuring a Yocto build environment, building a Rust application using Yocto\u0026rsquo;s native Rust support, cross-compiling the project, and deploying it to an embedded device.\nSetting Up the Yocto Build Environment (without kas) #The following steps show how to manually configure Yocto’s environment without relying on kas to build and run Rust programs on embedded Linux devices:\nYou can manually configure your Yocto environment by downloading the necessary repositories, setting up local.conf to enable Rust support, and initializing the build environment. If you prefer an automated setup, refer to the Appendix on using kas for automating the Yocto build environment.\nSetup Yocto as described here https://docs.yoctoproject.org/brief-yoctoprojectqs/index.html\nSet Up local.conf to Enable Rust Support Add the following lines to your local.conf file:\nENABLE_RUST = \u0026#34;1\u0026#34; IMAGE_FEATURES:append = \u0026#34; debug-tweaks ssh-server-openssh\u0026#34; Ensure Necessary Meta-layers Are Present in bblayers.conf Verify that the following meta-layers are included in your bblayers.conf:\nBBLAYERS ?= \u0026#34; \\ ${TOPDIR}/../meta-edge-controller \\ ${TOPDIR}/../meta-openembedded/meta-oe \\ ${TOPDIR}/../poky/meta \\ ${TOPDIR}/../poky/meta-poky \\ Building a Rust-based Application #With the Yocto environment set up, the next step is to create and compile your Rust application using Yocto\u0026rsquo;s built-in Rust compiler, without relying on the meta-rust layer. This approach simplifies the process by utilizing Yocto\u0026rsquo;s native Rust toolchain.\nIt is recommended to begin with a basic Rust project to ensure the initial setup works correctly. Once the basic application is successfully built, you can progressively introduce more dependencies and functionality.\nCreate a Rust Project Locally\nClone your Rust project repository:\ngit checkout https://github.com/\u0026lt;username\u0026gt;/\u0026lt;your-repo-name\u0026gt; For example: git clone https://github.com/konsulko/hello-rust\nUse devtool for Development\nAdd the project to Yocto’s workspace using devtool:\ndevtool add /path/to/\u0026lt;your-repo-name\u0026gt; Navigate to build/workspace/recipes/\u0026lt;your-repo-name\u0026gt; and modify the BitBake recipe as follows:\ninherit cargo cargo-update-recipe-crates For example: build/workspace/recipes/hello-rust\nThe cargo class allows Yocto to compile Rust programs by handling the integration of the Cargo build system, while the cargo-update-recipe-crates class enables automatic updates of the list of Cargo crates in SRC_URI by reading the Cargo.lock file.\nAt this point, you can remove any unnecessary tasks from the autogenerated BitBake file to keep it streamlined and focused.\nUpdate Cargo Dependencies\nRun the following command to generate the crates.inc file:\nbitbake -c update_crates \u0026lt;your-repo-name\u0026gt; Add the line require ${BPN}-crates.inc to your recipe.\nBuild and Test\nUse the devtool to build the project:\ndevtool build \u0026lt;your-repo-name\u0026gt; Add the project to the Yocto image by modifying local.conf:\nIMAGE_INSTALL:append = \u0026#34;\u0026lt;executable-name\u0026gt;\u0026#34; For example: IMAGE_INSTALL:append = \u0026quot;hello-rs\u0026quot;\nBuild the full image:\nbitbake core-image-minimal Cross-Compiling the Rust Application #Yocto’s built-in toolchain for Rust ensures that your project is compiled for the architecture specified in your local.conf or, if using kas, in the kas.yml configuration.\nYocto’s built-in toolchain for Rust ensures that your project is compiled for the architecture specified in the kas.yml configuration.\nDeploying to the Embedded Target #After building the application, the next step is to deploy it to your target embedded device.\nRun the Emulator\nOnce the image is built, you can run the emulator to test your Rust application in a virtual environment:\nrunqemu qemux86-64 nographic slirp Run the Application\nAfter booting the system, run your Rust application directly from the terminal:\n\u0026lt;executable-name\u0026gt; Note: Replace \u0026lt;executable-name\u0026gt; with the actual name of the binary you built, for example, hello-rs.\nBy using kas to manage your Yocto build environment and leveraging Yocto\u0026rsquo;s default Rust support without relying on meta-rust, you simplify the process of setting up and maintaining your build configurations. Combining Rust\u0026rsquo;s performance and safety with Yocto\u0026rsquo;s flexibility is an excellent choice for creating efficient and secure embedded applications.\nFor more context, this approach builds on the concepts outlined in Paul Barker\u0026rsquo;s original post on using Rust with Yocto, with the newest version of Yocto Scarthgap\nAppendix #Automating the Yocto Build Environment with kas # Install kas\nFirst, install kas on your development machine using pip:\npip install kas Create a kas Configuration File\nCreate a kas.yml configuration file with the necessary settings for a qemux86-64 machine:\nheader: version: 14 defaults: repos: branch: scarthgap machine: qemux86-64 local_conf_header: dev: | ENABLE_RUST = \u0026#34;1\u0026#34; IMAGE_FEATURES:append = \u0026#34; debug-tweaks ssh-server-openssh\u0026#34; storage: | INHERIT+=\u0026#34;rm_work\u0026#34; distro: poky repos: poky: url: https://github.com/yoctoproject/poky.git layers: meta: meta-poky: meta-openembedded: url: https://github.com/openembedded/meta-openembedded.git layers: meta-oe: target: - core-image-minimal Initialize the Build Environment\nUse kas to set up and enter the build environment:\nkas shell qemux86-64.yml -c bash This command sets up the build environment and opens a new shell where you can run BitBake commands.\nAfter initializing the build environment using kas, you can now proceed to building a Rust-based application.\nReferences # Using Rust with Yocto Project by Paul Barker Yocto Project Documentation Rust Cross-Compilation Guide kas Documentation ","date":"September 15, 2024","permalink":"/posts/yocto-rust/","section":"Posts","summary":"\u003cp\u003eIn embedded systems and industrial edge computing, using a reliable, efficient, and safe language like Rust is crucial\nfor developing applications where performance and memory safety are critical. This post presents how to develop\nRust-based applications using Yocto Linux\u0026rsquo;s default Rust compiler setup without relying on the \u003cstrong\u003e\u003ccode\u003emeta-rust\u003c/code\u003e\u003c/strong\u003e layer.\u003c/p\u003e\n\u003cp\u003e\n\n\n\n\n\n\n  \n  \n\u003cfigure\u003e\u003cimg src=\"/img/saad-salim-PqRvLsjD_TU-unsplash.jpg\" alt=\"industry\" class=\"mx-auto my-0 rounded-md\" /\u003e\n\u003c/figure\u003e\n\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eImage by Saad Salim from unsplash\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eThis guide focuses on configuring a Yocto build environment, building a Rust application using Yocto\u0026rsquo;s native Rust\nsupport, cross-compiling the project, and deploying it to an embedded device.\u003c/p\u003e","title":"Developing Rust Applications for Embedded Linux"},{"content":"","date":null,"permalink":"/categories/development/","section":"Categories","summary":"","title":"Development"},{"content":"","date":null,"permalink":"/tags/embedded-linux/","section":"Tags","summary":"","title":"Embedded Linux"},{"content":"","date":null,"permalink":"/categories/embedded-systems/","section":"Categories","summary":"","title":"Embedded Systems"},{"content":"","date":null,"permalink":"/tags/kas/","section":"Tags","summary":"","title":"Kas"},{"content":"","date":null,"permalink":"/categories/rust/","section":"Categories","summary":"","title":"Rust"},{"content":"","date":null,"permalink":"/tags/rust/","section":"Tags","summary":"","title":"Rust"},{"content":"","date":null,"permalink":"/tags/yocto/","section":"Tags","summary":"","title":"Yocto"},{"content":"Edge computing gateways serve as the pivotal nodes within the Industrial Internet of Things (IIoT) architecture, acting as critical intermediaries between industrial machinery and cloud-based systems. These devices facilitate local data processing and decision-making at the edge, playing a crucial role in the overall efficiency of industrial operations.\nHowever, a significant challenge persists: the effective management and diagnosis of these edge devices in field conditions, particularly when faced with limited resources and harsh environmental factors 1.\nImage by CHUTTERSNAP from unsplash\nThe Problem: Field Diagnostics for Edge Devices #Field technicians and machine operators frequently encounter difficulties in maintaining and troubleshooting edge devices. Traditional diagnostic methods, such as direct access via a serial console or secure shell (SSH) on a laptop, can be cumbersome and time-intensive. While remote access and fleet management tools offer scalable solutions, their effectiveness diminishes in scenarios where devices lack internet connectivity or face connectivity-related issues 2.\nTo overcome these challenges, the following strategies can be employed:\nHybrid Approach: Implement a combined strategy of remote access and local troubleshooting. Remote tools can be utilized for initial diagnostics and configuration, while technicians are equipped with portable devices or tools for on-site interventions. Enhanced Connectivity: Investigate alternative connectivity solutions such as cellular modems, satellite links, or mesh networks to maintain reliable communication with edge devices. Local Diagnostics Tools: Equip technicians with specialized on-site diagnostic tools or software capable of isolating and addressing issues independently of full internet connectivity. Cloud-Based Data Storage: Utilize cloud storage for critical device data and logs, enabling remote access and analysis even under limited connectivity conditions. Automated Troubleshooting: Deploy automated scripts or AI-driven tools for identifying and resolving common issues with greater efficiency. By adopting these strategies, organizations can enhance the efficiency and effectiveness of edge device maintenance and troubleshooting, even in the most challenging environments.\nAddressing technical challenges #Field diagnostics for edge devices involve navigating a multitude of technical challenges that can significantly impact the success of diagnostic efforts. Each challenge must be carefully considered when selecting and implementing maintenance strategies.\nHarsh Industrial Environments: Industrial environments are characterized by extreme conditions such as dust, moisture, temperature fluctuations, and vibrations, all of which can adversely affect the performance of diagnostic tools. Devices operating in these environments must be robust enough to maintain reliable functionality despite these challenges 3.\nResource-Constrained Devices: Edge devices often have limited computational resources, including processing power and memory, necessitating the use of lightweight and efficient diagnostic tools. These devices may also rely on constrained power supplies and may lack consistent network connectivity, further complicating diagnostic processes 4.\nComplex System Interactions: Edge Linux devices require a deep understanding of the Linux operating system, which local operators may not possess. This complexity is increased when diagnostics are performed through interfaces like serial consoles, where specialized knowledge is required 5.\nEase of Use in the Field: Technicians require diagnostic tools that are both quick to use and easy to navigate. In field conditions, especially when accessing devices via serial console or SSH, the user interface must be intuitive to minimize the time and effort needed for diagnostics [^11].\nCurrent Approaches and Their Limitations #There are current diagnostic systems that depend on graphical user interfaces (GUIs) or web-based dashboards, which, while effective in controlled environments, pose significant limitations in field applications 6:\nResource Intensity: GUIs often require more processing power and memory, which can be a limitation for edge devices.\nEnvironmental Sensitivity: GUIs are less practical in harsh field conditions where screen readability can be compromised, and industrial touch screens might not be practical, as it might be too hot or too cold for reliable operation.\nUser Complexity: Navigating through multiple graphical screens can be time-consuming and cumbersome for technicians.\nChoosing the Right Interface #The choice of diagnostic interface should be guided by the specific requirements of the application. Text-based interfaces are particularly advantageous in field diagnostics due to their simplicity, resource efficiency, and resilience in harsh conditions. These interfaces are ideal for straightforward processes and serial interactions, enabling experts to conduct diagnostics swiftly and accurately, even with minimal tools 7.\nConversely, GUIs or web-based dashboards may be more appropriate in environments with sufficient processing power, favorable environmental conditions, and a need for complex data visualization.\nUltimately, the interface selection should align with the operational context and the specific requirements of the application, particularly in remote fleet-managed systems 8.\nUse Case: Diagnosing Edge Devices in a Remote Industrial Facility #Consider a scenario where a large manufacturing facility operates in a remote, harsh environment, such as a mining site or an offshore oil rig. The facility relies heavily on a network of IIoT edge devices to monitor and control critical operations. These devices must operate reliably despite exposure to dust, moisture, extreme temperatures, and constant vibrations.\nIn such environments, when an edge device malfunctions or experiences connectivity issues, deploying a technician to diagnose the problem is both costly and time-consuming. The technician must be capable of quickly accessing the device, identifying the issue, and implementing a solution while contending with a lack of local expertise, environmental constraints, and limited device resources.\nChoosing the Right Diagnostic Tools #To address these challenges, diagnostic tools must be carefully selected to meet the environmental and operational constraints of the field. Text-based interfaces, for example, are particularly well-suited for these scenarios due to their simplicity, resource efficiency, and resilience in harsh conditions. These interfaces allow for rapid and accurate diagnostics, even with minimal tools and through direct access methods 7.\nExamples of Diagnostic Applications #These examples underscore the importance of selecting diagnostic tools and interfaces that are tailored to the specific operational needs and constraints of the application, ensuring that edge devices remain reliable and efficient even in the most demanding industrial environments.\nSoftware Installer: Specialized installer software can streamline the setup, configuration, and troubleshooting of edge devices in the field. This is particularly valuable in scenarios requiring rapid deployment and reconfiguration, reducing the time and effort needed to maintain operational efficiency. Field Diagnostics: Text-based diagnostic tools provide quick assessments and issue resolution for devices operating in remote or harsh environments, where GUI-based tools may be impractical due to resource constraints or environmental conditions. Remote Troubleshooting: In cases where remote access is possible but challenging due to connectivity issues, lightweight web-based dashboards can offer a balance between user-friendliness and resource efficiency, enabling technicians to manage multiple devices across a fleet In conclusion, the field diagnostics of edge devices in industrial environments present unique challenges that require a thoughtful and tailored approach. Effective solutions must be rugged, resource-efficient, and carefully aligned with the specific conditions and requirements of the application. By considering these factors, organizations can enhance the reliability and efficiency of their edge computing infrastructure, ensuring optimal performance in even the most challenging environments.\nMcKinsey, The Internet of Things: Catching Up to an Accelerating Opportunity\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nMcKinsey, Unlock Value with an Industrial IoT Technology Stack That Scales\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nAnalog, Intelligent Field Instruments: The Smart Way to Industry 4.0\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nMDPI, Challenges for Industrial IoT\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nACM, Navigating Linux Environments in Edge Devices\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nSelectHub, IoT Transforming Field Service Management\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nSCAND, Creating a UI for an IoT Application: Things to Consider\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nMDPI, The Importance of Diagnostic Tools in Industrial IoT\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"September 11, 2024","permalink":"/posts/industrial_edge_computing_field_interaction/","section":"Posts","summary":"\u003cp\u003eEdge computing gateways serve as the pivotal nodes within the Industrial Internet of Things (IIoT) architecture, acting\nas critical intermediaries between industrial machinery and cloud-based systems. These devices facilitate local data\nprocessing and decision-making at the edge, playing a crucial role in the overall efficiency of industrial operations.\u003c/p\u003e\n\u003cp\u003eHowever, a significant challenge persists: the effective management and diagnosis of these edge devices in field\nconditions, particularly when faced with limited resources and harsh environmental factors \u003csup id=\"fnref:1\"\u003e\u003ca href=\"#fn:1\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e1\u003c/a\u003e\u003c/sup\u003e.\u003c/p\u003e","title":"Enhancing Industrial IoT with Robust Edge Computing Diagnostics"},{"content":"Kubernetes, as a container orchestration platform, has revolutionized the way we deploy and manage applications. While Kubernetes offers a robust framework for running containers at scale, managing complex applications that serve millions of users can be an intimidating task.\nNowadays, is not often that we run applications on only one server. Today, applications are made up of many small parts that can grow or shrink as needed running distributed in several machines or services, locally or in the cloud. Managing all these components can become difficult without automation, which is why self-healing apps are becoming a necessity for operations teams.\nImage by Pexels from Pixabay\nWhat is a Kubernetes Operator? #A Kubernetes Operator is a method of packaging, deploying, and managing applications on Kubernetes clusters with automation in mind. They are software components that automate tasks such as configuration, updates, scaling, and recovery.\nOperators use a set of Kubernetes APIs and patterns to handle the entire lifecycle of a particular application or service. They are especially useful when dealing with stateful applications or services that require more than basic deployment and scaling logic.\nOne of the most significant advantages of Kubernetes Operators is their ability to manage applications across multiple clusters, making them valuable assets for scenarios like multi-cloud deployments or hybrid cloud setups.\nThe core components of a Kubernetes Operator are:\nCustom Resource Definitions (CRDs): These extend the Kubernetes API to define new types of resources specific to your application. CRDs allow you to define the structure and behavior of your application within the Kubernetes ecosystem.\nController Logic: The controller watches for changes to custom resources (CRs) created based on the defined CRDs. When a CR is created, updated, or deleted, the controller reconciles the actual state of the application with the desired state defined in the CR.\nReconciliation Loop: The reconciliation loop continuously compares the current state of the application with the desired state defined in the CR. If there is a discrepancy, the operator takes the necessary actions to align the two states.\nCustom Logic: Operators often include custom logic of the specific application being managed. This can involve tasks such as application-specific scaling, data migration, backup and restore, and more.\nLearn more about Kubernetes Operators in this Kubernetes Operators Explained guide.\nImplementation #To implement Kubernetes Operators, there are multiple frameworks available. One notable option is the Operator Framework SDK, which provides open-source libraries for creating operators using Go, Ansible, Helm, and, as of April 2023, Java with the Java Operator SDK.\nIntroducing the Java Operator SDK #This framework empowers developers to build Kubernetes Operators using Java. It\u0026rsquo;s especially convenient for platforms developed using Java or any of the JVM languages. You can learn more about it in this Red Hat article.\nSome key benefits of using the Java Operator SDK for managing your Kubernetes applications are:\nAutomation: The Java Operator SDK abstracts away much of the complexity of interacting with the Kubernetes API, allowing you to focus on defining how your application should behave.\nConsistency: With operators, you ensure that your application\u0026rsquo;s behavior is consistent across different environments, which is invaluable in complex microservices architectures.\nScalability: Operators are designed to work well with large-scale applications. As your application grows, the operator can handle the increased load and management tasks.\nCustomization: You have full control over how your operator behaves and how it interacts with your application.\nKubernetes Operators, coupled with the Java Operator SDK, supply a powerful toolkit for simplifying the management of complex applications in Kubernetes clusters. They bring automation, consistency, scalability, and customization to the forefront, ensuring that your applications run smoothly in any Kubernetes environment.\n","date":"December 26, 2023","permalink":"/posts/develop_kubernetes_operators/","section":"Posts","summary":"\u003cp\u003eKubernetes, as a container orchestration platform, has revolutionized the way we deploy and manage applications. While\nKubernetes offers a robust framework for running containers at scale, managing complex applications that serve millions\nof users can be an intimidating task.\u003c/p\u003e\n\u003cp\u003eNowadays, is not often that we run applications on only one server. Today, applications are made up of many small parts\nthat can grow or shrink as needed running distributed in several machines or services, locally or in the cloud. Managing\nall these components can become difficult without automation, which is why self-healing apps are becoming a necessity\nfor operations teams.\u003c/p\u003e","title":"Developing Kubernetes Operators"},{"content":"Event-driven systems efficiently handle streams of events or data by processing them as they occur. An event, defined as any observable occurrence at a specific point in time forms the core of these systems1. Due to the decoupled nature of event-driven architectures, effective communication of the events content and intent becomes crucial, relying on both proper semantics and syntax.\nEffective communication, whether written or spoken, requires careful attention to syntax and semantics2. Messages with proper syntax are not only easier to read or listen to but also pave the way for clear semantics, enabling an accurate understanding of the intended meaning and fostering successful communication between sources and receivers.\nPhoto by Mathyas Kurmann on Unsplash\nData Modeling #From ancient times, mapping has served as a means for people to communicate across different languages. In the realm of software engineering, this mapping is evident in various tools and techniques such as Model Driven Engineering ( MDE) 3, Unified Modeling Language (UML) 4, and Domain-Driven Engineering (DDD) 5.\nIn Geographical Mapping, a map symbolically represents selected characteristics of a place, reflecting the real world 6. Within Domain-Driven Design, the Domain Model organizes knowledge and data around the problem context, capturing the concepts of the problem domain 5. When creating a message structure, it is crucial to provide a comprehensive description to prevent ambiguity and duplication. The message should exhibit a clear knowledge structure ( syntax) and strive to convey the intended meaning (semantic). However, similar to mapping, including too many details in the message can increase its size, diminishing readability while augmenting elements that aid in understanding different locations on the map.\nMessage Exchange #Exchanging messages between people usually is not an easy feat. Let\u0026rsquo;s forget about security and Byzantine problems for a minute. Assume two persons speak English, and they want to send a letter via post. What do you need?\nAddress and person to whom you are sending the message Address and person who is sending the message and expects an answer What else do you need? The letter content, easy right? Unfortunately, no. You need clear language and grammar at a level both persons understand. How about if you are sending specific content like a measure, temperature, time, and location? You need to specify the metric system, time zone, and coordinate system.\nNow, let\u0026rsquo;s say we have a good enough message system for 100 persons, sending the same message. After a while, we are sending new data, such as distance. Now you have to make sure everyone is clear with all the rules specified before and what has changed.\nUsually, between people, it\u0026rsquo;s a manageable process. But if you include machines in the process of exchanging messages, and it\u0026rsquo;s not 100 persons but millions of devices, this becomes an unmanaged problem, given the variability of the world and the models used for communication.\nTackling the Issues #Extensive research has been done to address this challenge, offering solutions ranging from Object-Oriented typed languages to the frameworks of Service-Oriented Architectures and Microservices. In today\u0026rsquo;s landscape, these solutions commonly use versatile exchange formats, including Extensible Markup Language (XML), JavaScript Object Notation (JSON), and YAML.\nThese data formats are pivotal, acting as a bridge between human understanding and machine processing 7. While XML excels in enforcing precise rules8, its verbosity prompts consideration of more streamlined alternatives such as JSON and YAML.\nA scenario where everyone on a team speaks the same language, a language shared by both business and tech experts. This is precisely what DDD advocates for—a shared vocabulary or \u0026ldquo;ubiquitous language.\u0026rdquo; Moreover, DDD recommends the use of bounded contexts, breaking down the project into manageable segments. This minimizes confusion and prevents teams from being inundated with unnecessary intricacies.\nThe significance of this approach becomes apparent when working collaboratively. A shared understanding and organized structure make the project more manageable, akin to having a well-defined map for the entire journey. DDD facilitates consensus on how components should function and their desired appearance—a fundamental aspect of ensuring the success of any project.\nTo navigate this process effectively, it\u0026rsquo;s imperative to maintain synchronization across diverse teams collaborating on the project. Establishing agreement on both semantics and syntax becomes the foundation of a successful implementation.\nConclusion #In the world of event-driven systems, effective communication plays a crucial role. Events, the main actors in these systems, require a blend of clear language and understanding to function well. We\u0026rsquo;ve explored data modeling, drawing comparisons to ancient map-making tools used in today\u0026rsquo;s software landscape.\nWhether it\u0026rsquo;s sending letters between people or managing messages among millions of devices, we\u0026rsquo;ve seen that clear communication faces challenges. Despite advanced tech solutions, the everyday challenge for engineers is navigating the twists and turns of communication in our digital era. Thus, the journey goes on, highlighting that simple and effective communication is the key to success for every engineer.\nReferences # Event Streams in Action: Real-time event systems with Kafka and Kinesis\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nPrinciples of Verbal Communication\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nModel Driven Engineering\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nWhat is Unified Modeling Language (UML)?\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nWhat is the Domain Model in Domain Driven Design?\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nMap\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nWhat is XML?\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nXML introduction\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"December 9, 2023","permalink":"/posts/event_driven_message_exchange/","section":"Posts","summary":"\u003cp\u003eEvent-driven systems efficiently handle streams of events or data by processing them as they occur. An event, defined as\nany observable occurrence at a specific point in time forms the core of these systems\u003csup id=\"fnref:1\"\u003e\u003ca href=\"#fn:1\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e1\u003c/a\u003e\u003c/sup\u003e. Due to the decoupled nature\nof event-driven architectures, effective communication of the events content and intent becomes crucial, relying on both\nproper semantics and syntax.\u003c/p\u003e\n\u003cp\u003eEffective communication, whether written or spoken, requires careful attention to syntax and semantics\u003csup id=\"fnref:2\"\u003e\u003ca href=\"#fn:2\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e2\u003c/a\u003e\u003c/sup\u003e. Messages\nwith proper syntax are not only easier to read or listen to but also pave the way for clear semantics, enabling an\naccurate understanding of the intended meaning and fostering successful communication between sources and receivers.\u003c/p\u003e","title":"From Data Modeling to Effective Message Exchange"},{"content":"Harnessing the Power of Rust for Microcontroller Projects with the Raspberry Pi Pico\nIntroduction #The Internet of Things (IoT) and microcontroller development are areas that bring great opportunities for creators, letting you interact with the physical world in new and imaginative ways. Rust, as a programming language, brings its own strengths to the table, such as speed and safety, which are essential for turning a good idea into a working IoT product.\nThis guide is all about getting you up and running quickly with a Rust project on the Raspberry Pi Pico. It will show you how to set up your project, and lead you through the practical steps of flashing and debugging your code with the Raspberry Debug Probe. It\u0026rsquo;s a straightforward path to bringing your IoT project to life.\nPrerequisites #Prepare your environment, and let\u0026rsquo;s dive into the exciting world of Rust and the Raspberry Pi Pico. We have leverage the rp2040 HAL (Hardware Abstraction Layer) with the rp-pico BSP (Board Support Package), which abstracts the complexities of the RP2040 microcontroller.\nEnsure Rust and Cargo are installed on your development machine. Follow the official installation instructions. Obtain a Raspberry Pi Pico board RP2040 to complete this project. Acquire the Raspberry Pi Debug Probe for a streamlined development process. This project can be executed from various platforms, including Raspberry Pi 4, Windows PC, macOS, or Linux. A Raspberry Pi 4 is used in this guide. Getting Started #Setting up an embedded Rust project for the Raspberry Pi Pico involves 3 general steps:\n1. Environment Preparation # Hardware Setup: Connect the Raspberry Debug Probe to the Raspberry Pi Pico as outlined in the Raspberry Pi Debug Probe documentation.\nWiring Guide\nDevelopment Environment Setup: Verify Rust and Cargo installations with rustc --version.\n2. Debug Probe and SWD Connection Configuration # Install the necessary tools for deployment and debugging.\nsudo apt install libssl-dev libudev-dev openocd gdb-multiarch rustup self update rustup update stable rustup target add thumbv6m-none-eabi Test the debug probe connectivity.\nsudo openocd -f interface/cmsis-dap.cfg -f target/rp2040.cfg -c \u0026#34;adapter speed 5000\u0026#34; Configure udev rules to allow non-root access to the Raspberry Pi Debug Probe. For a detailed guide, see Getting Started with Rust on a Raspberry Pi Pico.\na. Get the necessary information for PID and VID using lsusb | grep Probe. The output should look like this:\nBus 001 Device 003: ID 2e8a:000c Raspberry Pi Debug Probe (CMSIS-DAP) The PID is 2e8a, and the VID is 000c.\nb. Add a file to the udev rules, for example:\nsudo vim /etc/udev/rules.d/99-raspberrypi-debug-probe.rules c. Append the following two lines to the end:\n# Raspberry Pi Debug Probe ATTRS{idVendor}==\u0026#34;2e8a\u0026#34;, ATTRS{idProduct}==\u0026#34;000c\u0026#34;, MODE:=\u0026#34;0666\u0026#34; The 0666 permissions allow all users to read and write but not execute the device nodes.\nd. Restart the service with sudo service udev restart.\n3. Rust Project Setup and Compilation # Use the Raspberry Pi Pico Project Template to create a new repository.\nClone the new project into you development machine.\nGo to the new project and get the dependencies and build it\ncargo install elf2uf2-rs --locked cargo install flip-link cargo install probe-rs --features=cli --locked Compile and flash the project using cargo run. Optionally, for a Raspberry Pico W, connect an external LED to a GPIO pin for visual feedback, such as GP 13. Include the following lines:\nlet mut light_pin = pins.gpio13.into_push_pull_output(); loop { ... light_pin.set_high().expect(\u0026#34;Cannot turn the LED on\u0026#34;); ... light_pin.set_low().expect(\u0026#34;Cannot turn the LED off\u0026#34;); ... } Refer to the Raspberry Pi Pico Pinout documentation for more details.\nConclusion #As we wrap up, we\u0026rsquo;ve managed to give the Raspberry Pi Pico new abilities with Rust—flashing it with new code, debugging on the fly, and controlling things like lights and sensors. This project it\u0026rsquo;s a glimpse into a future where using Rust for microcontrollers can lead to even more clever and unique tech creations.\nThere are numerous guides to get you started with Rust and the Pico board, offering various methods to achieve similar outcomes. Whether it\u0026rsquo;s simply using the Pico as a USB device, connecting it to a Raspberry Pi SoC, or utilizing another Pico as a debug probe, each approach has its advantages, allowing you to choose what works best for you.\nLooking forward, as we dive deeper into Rust and its use in microcontrollers and SoC\u0026rsquo;s, we\u0026rsquo;re sure to find even more exciting ways to innovate and build.\nReferences # Official Rust Installation Instructions Raspberry Pi Pico board Raspberry Pi Debug Probe rp2040 HAL Raspberry Pi Debug Probe Documentation Raspberry Pi Pico Project Template Getting Started with Rust on a Raspberry Pi Pico ","date":"November 7, 2023","permalink":"/posts/embedded_rust_pico/","section":"Posts","summary":"\u003cp\u003e\u003cstrong\u003e\u003cem\u003eHarnessing the Power of Rust for Microcontroller Projects with the Raspberry Pi Pico\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n\u003ch2 id=\"introduction\" class=\"relative group\"\u003eIntroduction \u003cspan class=\"absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100\"\u003e\u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700\" style=\"text-decoration-line: none !important;\" href=\"#introduction\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\u003c/span\u003e\u003c/h2\u003e\u003cp\u003eThe Internet of Things (IoT) and microcontroller development are areas that bring great opportunities for creators,\nletting you interact with the physical world in new and imaginative ways. Rust, as a programming language, brings its\nown strengths to the table, such as speed and safety, which are essential for turning a good idea into a working IoT\nproduct.\u003c/p\u003e","title":"Developing with Rust and the Raspberry Pi Pico"},{"content":"","date":null,"permalink":"/tags/microcontroller/","section":"Tags","summary":"","title":"Microcontroller"},{"content":"","date":null,"permalink":"/tags/raspberry-pi-pico/","section":"Tags","summary":"","title":"Raspberry-Pi-Pico"},{"content":"A few years ago, I had reservations about the project\u0026rsquo;s maturity and its future prospects. Today, having earned my first Rust certificate, I\u0026rsquo;m excited to apply this knowledge to microprocessors and embedded Linux devices. Rust, a relatively new programming language, has rapidly gained popularity, particularly in system programming, thanks to its vibrant community.\nNow that Rust has made its way into the Linux Kernel, offers robust support for Yocto Linux, and boasts various HAL crates for bare-metal embedded development, it\u0026rsquo;s a compelling language to explore and master.\nIn this post, I\u0026rsquo;ll show you why Rust might interest you as well.\nPhoto by Rod Long on Unsplash\nWhat is Rust? #Rust is the most admired language as of 2023, according to the latest Stack Overflow survey1.\nRust is a modern systems programming language with a focus on performance, concurrency, and memory efficiency2. Created by Mozilla, it offers a unique combination of low-level control over hardware and high-level abstractions for writing safe and concurrent code. The language is known for its memory safety guarantees and zero-cost abstractions.\nThe Rust community is rapidly evolving the language, expanding its ecosystem, and addressing any issues or bugs that arise3. Rust has gained popularity in various domains, including system programming, web development, and more. The language is well-supported by a robust compiler and package manager, Cargo, making it easier for developers to create, share, and manage Rust projects.\nHowever, don\u0026rsquo;t expect Rust to replace existing projects developed in C or C++, like the Linux Kernel4. Rust will help you develop new projects or integrate into old ones, and it has a promising future with the Rust Foundation supporting its development5.\nWhy Use Rust for System Programming? #Rust\u0026rsquo;s appeal for system programming lies in its key features:\nMemory Safety: Rust uses a strict ownership system that prevents common programming errors like null pointer dereferences and data races, ensuring program safety. Concurrent Programming: Rust\u0026rsquo;s ownership model enables safe and efficient concurrent programming, allowing multithreading and parallelism without data race risks. Performance: Rust offers similar performance to languages like C and C++ while maintaining safety. It compiles to efficient machine code, making it suitable for system-level tasks. Ecosystem: Rust boasts a growing ecosystem of libraries and tools, facilitating work on a wide range of projects, from operating systems to web servers. Community: The Rust community is known for its support and collaborative nature, with well-maintained documentation and resources, making it easier for developers to learn and grow. In addition to its competence in system programming, Rust has also gained significant importance in the realm of embedded development5. Its memory safety guarantees and efficient performance make it an ideal choice for developing software on resource-constrained embedded systems as the Internet of Things (IoT) continues to grow.\nWho uses Rust? #Rust has gained adoption in various projects and companies, thanks to its unique set of advantages. Some notable examples include:\nMozilla: The organization behind Firefox played a pivotal role in Rust\u0026rsquo;s development and continues to use it for various projects, including parts of the Firefox browser. Amazon: Amazon Web Services (AWS) has started using Rust for some services, leveraging its safety and performance benefits. Microsoft: Microsoft is exploring Rust for system-level programming and security-critical projects, recognizing its potential in these areas. Red Hat: Red Hat, a prominent name in the open-source software world, is actively involved in the Rust community and is exploring its use in their products. These are just a few examples, and the Rust ecosystem continues to grow as more companies and developers recognize its potential in a wide range of applications, including system programming, embedded development, machine learning, data processing, and other cutting-edge technologies.\nConclusion #Earning my Rust certificate has been a rewarding experience, and I\u0026rsquo;m excited to continue my journey with this powerful and innovative programming language. Rust\u0026rsquo;s commitment to safety and performance is reshaping the landscape of system programming, embedded development, and beyond. I can\u0026rsquo;t wait to see what the future holds for Rust and its ever-expanding community.\nReferences # Stack Overflow Survey 2023\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nRust by Example\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nHow Rust went from a side project to the world’s most-loved programming language\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nRust Foundation\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nWhy Rust is a great fit for embedded software\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"October 14, 2023","permalink":"/posts/rust_programming_language/","section":"Posts","summary":"\u003cp\u003eA few years ago, I had reservations about the project\u0026rsquo;s maturity and its future prospects. Today, having earned my first\nRust certificate, I\u0026rsquo;m excited to apply this knowledge to microprocessors and embedded Linux devices. Rust, a relatively\nnew programming language, has rapidly gained popularity, particularly in system programming, thanks to its vibrant\ncommunity.\u003c/p\u003e\n\u003cp\u003eNow that Rust has made its way into the Linux Kernel, offers robust support for Yocto Linux, and boasts various HAL\ncrates for bare-metal embedded development, it\u0026rsquo;s a compelling language to explore and master.\u003c/p\u003e","title":"Getting to Know Rust"},{"content":"Edge computing is revolutionizing data processing by bringing computation closer to the source of data generation. It offers benefits like reduced latency and improved real-time processing but comes with significant challenges that require careful design.\nPhoto by Justin Luebke on Unsplash\nWhen considering an edge solution, context is everything. Imagine not just one device, but potentially thousands or even millions. These devices may operate unsupervised for years without recalls. The consequences of a failure, whether it\u0026rsquo;s just one device or many, can be significant - from downtime measured in hours to days or even months. Factor in potential issues like power failures, communication loss, or server maintenance, and you\u0026rsquo;ll begin grasping the complexity of such system.\nThe list of challenges of Edge Computing is long, however, the benefits for companies outweigh the challenges that need to be overcome. In this post, we\u0026rsquo;ll outline some of the common challenges to implement or develop an edge computing solution.\nSecurity #Imagine an edge device located in a remote area. It\u0026rsquo;s accessible physically, making it vulnerable to tampering. If it gets compromised, it could inject malicious code into your network, posing serious security risks. Robust security measures, such as encryption and access controls, are essential to prevent such incidents.\nOperational Resilience #Think about an edge device placed in a location where temperatures can soar running the cpu usage high. If it overheats or malfunctions, it could disrupt critical operations. Ensuring these devices are designed to withstand expected conditions is crucial to maintain operational resilience and minimize costly downtime.\nDevice Management #Picture managing thousands of edge devices spread across various regions. If a device fails during a software update and is not able to recover, it could result in a logistical nightmare. Having a robust device management system in place streamlines tasks like updates and troubleshooting, ensuring efficient operations even at scale.\nData Management #Consider data generated by edge devices with varying software versions and schemas. If these messages don\u0026rsquo;t align, data processing becomes chaotic, leading to errors and inefficiencies. Establishing a data management strategy that enforces consistent schemas and efficient data processing is vital.\nDistributed Communication #Envision an edge device sending critical data over an unreliable network. If the data doesn\u0026rsquo;t reach its destination due to network failures, it can disrupt operations. Implementing efficient and fault-tolerant communication protocols, including error correction, ensures data reaches its destination reliably.\nConclusion #Edge computing offers a number of benefits for real-time data processing and latency reduction. However, it is important to be aware of and address the technical challenges associated with edge computing before implementing a solution. By carefully planning, optimizing, and implementing an edge solution, organizations can reap the benefits of edge computing while minimizing the risks and costs that could incur on a failure scenario.\n","date":"October 3, 2023","permalink":"/posts/edge_computing_challenges/","section":"Posts","summary":"\u003cp\u003eEdge computing is revolutionizing data processing by bringing computation closer to the source of data generation. It\noffers benefits like reduced latency and improved real-time processing but comes with significant challenges that\nrequire careful design.\u003c/p\u003e\n\u003cp\u003e\n\n\n\n\n\n\n  \n  \n\u003cfigure\u003e\u003cimg src=\"/img/justin-luebke-BkkVcWUgwEk-unsplash.jpg\" alt=\"Edge Computing\" class=\"mx-auto my-0 rounded-md\" /\u003e\n\u003c/figure\u003e\n\u003c/p\u003e\n\u003cp\u003e\u003cem\u003ePhoto by Justin Luebke on Unsplash\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eWhen considering an edge solution, context is everything. Imagine not just one device, but potentially thousands or even\nmillions. These devices may operate unsupervised for years without recalls. The consequences of a failure, whether it\u0026rsquo;s\njust one device or many, can be significant - from downtime measured in hours to days or even months. Factor in\npotential issues like power failures, communication loss, or server maintenance, and you\u0026rsquo;ll begin grasping the\ncomplexity of such system.\u003c/p\u003e","title":"Edge Computing Implementation Challenges"},{"content":"Apache Kafka, a distributed streaming platform, is a valuable tool for handling data streams. In this guide, we\u0026rsquo;ll walk through the steps to implement a Kafka producer in C++. This will enable you to efficiently send data to a Kafka topic and integrate it into your data processing pipelines.\nImage by Benjamin Zocholl from Pixabay\nTo illustrate the process, the project kafka-producer contains the source code.\nSetting Up Your Development Environment #Before you start coding, ensure you have the necessary tools and libraries installed, including a C++ compiler and the librdkafka C/C++ library, which is a popular choice for Kafka integration in C++. You can install it via package managers or compile it from source. We will also use CMake, GTest, and GMock installed on an Ubuntu 22.04 LTS System to provide a rock-solid foundation for further development of your Producer.\nInstall the required libraries to your Ubuntu machine: sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y \\ git \\ build-essential \\ cmake \\ clang \\ libcurl4-openssl-dev \\ liblz4-dev \\ zlib1g-dev \\ libssl-dev \\ libsasl2-dev Download the latest version of librdkafka LIBRDKAFKA_VERSION=\u0026#34;v2.2.0\u0026#34; \u0026amp;\u0026amp; git clone --branch $LIBRDKAFKA_VERSION \\ https://github.com/confluentinc/librdkafka.git librdkafka Compile the library for cmake and installed on the machine cd librdkafka cmake -H. -B_cmake_build cmake --build build -j 6 cmake --build _cmake_build --target install The Docker File can be used to help reproduce and have the compiled library.\nCreating a Kafka Producer #To build a Kafka producer in C++, you\u0026rsquo;ll need to use the librdkafka library.\nHere are the basic steps of how to create a producer:\nCreate a configuration to connect to a broker\nstd::unique_ptr\u0026lt;RdKafka::Conf\u0026gt; conf(RdKafka::Conf::create(RdKafka::Conf::CONF_GLOBAL)); conf-\u0026gt;set(\u0026#34;bootstrap.servers\u0026#34;, brokersList, error_str); Create a Kafka producer\nstd::unique_ptr\u0026lt;RdKafka::Producer\u0026gt; producer(RdKafka::Producer::create(conf.get(), error_str)); Send a message\nRdKafka::ErrorCode producerError = kafkaProducer-\u0026gt;produce( topic, RdKafka::Topic::PARTITION_UA, RdKafka::Producer::RK_MSG_COPY, const_cast\u0026lt;char *\u0026gt;(message.c_str()), message.size(), /* Key */ NULL, 0, /* Timestamp (defaults to current time) */ 0, /* Message headers, if any */ NULL, /* Per-message opaque value passed to * delivery report */ NULL ); The complete code for the producer can be found here and the test here. This code sets up a Kafka producer, sends a message to a specified topic, and handles any potential errors.\nConclusion #By following this guide, you\u0026rsquo;ve learned how to implement a Kafka producer in C++ using the librdkafka library. This producer can be a valuable component in your data processing pipelines, enabling you to efficiently send data to Kafka topics and integrate it into your real-time data processing workflows.\n","date":"September 22, 2023","permalink":"/posts/implement_kafka_cpp_producer/","section":"Posts","summary":"\u003cp\u003eApache Kafka, a distributed streaming platform, is a valuable tool for handling data streams. In this guide, we\u0026rsquo;ll walk\nthrough the steps to implement a Kafka producer in C++. This will enable you to efficiently send data to a Kafka topic\nand integrate it into your data processing pipelines.\u003c/p\u003e\n\u003cp\u003e\n\n\n\n\n\n\n  \n  \n\u003cfigure\u003e\u003cimg src=\"/img/river-6068374_640.jpg\" alt=\"blocks\" class=\"mx-auto my-0 rounded-md\" /\u003e\n\u003c/figure\u003e\n\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eImage by Benjamin Zocholl from Pixabay\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eTo illustrate the process, the project \u003ca href=\"https://gitlab.com/advt3/kafka-producer/-/tree/v0.0.1\" target=\"_blank\" rel=\"noreferrer\"\u003ekafka-producer\u003c/a\u003e contains the source\ncode.\u003c/p\u003e\n\u003ch2 id=\"setting-up-your-development-environment\" class=\"relative group\"\u003eSetting Up Your Development Environment \u003cspan class=\"absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100\"\u003e\u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700\" style=\"text-decoration-line: none !important;\" href=\"#setting-up-your-development-environment\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\u003c/span\u003e\u003c/h2\u003e\u003cp\u003eBefore you start coding, ensure you have the necessary tools and libraries installed, including a C++ compiler and the\nlibrdkafka C/C++ library, which is a popular choice for Kafka integration in C++. You can install it via package\nmanagers or compile it from source. We will also use CMake, GTest, and GMock installed on an Ubuntu 22.04 LTS System to\nprovide a rock-solid foundation for further development of your Producer.\u003c/p\u003e","title":"Implementing a Kafka Producer in C++"},{"content":"Numerous companies aim to understand human behavior and its context by collecting and analyzing real or near real-time events. Apache Kafka is the predominant solution for this purpose, preferred by a vast number of businesses, including \u0026quot; over 80% of the Fortune 100\u0026quot; [1]. Initially developed by LinkedIn to understand human interests and facilitate prompt responses, Kafka was made open-source in 2011.\nImage by Benjamin Zocholl from Pixabay\nKafka\u0026rsquo;s Strengths #Kafka excels in managing massive data volumes with minimal latency and high throughput. It achieves this through:\nHorizontal scalability to adapt to growing data loads swiftly. Ensuring durability via replication and fault-tolerant mechanisms. Enforcing exactly-once message processing. Unique Aspects of Kafka #What sets Kafka apart is its rich ecosystem, vibrant community, and the option for official support from **Confluent.io **. The co-founders of Kafka created it during their time at LinkedIn in 2008.\nKey Components of Apache Kafka #Kafka comprises the following essential elements:\nTopics: Logical data stream categories. Brokers: Machines managing data storage, distribution, and processing. Partitions: Divisions of topics for parallelism and load balancing. Producers: Initiators of data streams, sending messages to topics. Consumers: Subscribers processing streamed data. Consumer Groups: Logical sets of consumers for parallel processing. Connectors: Facilitators for data movement between Kafka and external systems. Kafka Streams: A library for building real-time stream processing apps. Kafka\u0026rsquo;s Role in the Data-Driven World #In a data-centric world, Kafka stands as an innovation cornerstone. Its architecture enables efficient, fault-tolerant data streaming, empowering businesses to make real-time decisions. From scalability to ecosystem support, Kafka drives data-driven transformations, delivering unparalleled speed, reliability, and insights.\nReferences # [1]https://kafka.apache.org Kafka: The Definitive Guide Kafka in Action ","date":"August 23, 2023","permalink":"/posts/kafka_stream_processing_introduction/","section":"Posts","summary":"\u003cp\u003eNumerous companies aim to understand human behavior and its context by collecting and analyzing real or near real-time\nevents. Apache Kafka is the predominant solution for this purpose, preferred by a vast number of businesses, including \u0026quot;\nover 80% of the Fortune 100\u0026quot; \u003ca href=\"https://kafka.apache.org\" target=\"_blank\" rel=\"noreferrer\"\u003e[1]\u003c/a\u003e. Initially developed by LinkedIn to understand human\ninterests and facilitate prompt responses, Kafka was made open-source in 2011.\u003c/p\u003e\n\u003cp\u003e\n\n\n\n\n\n\n  \n  \n\u003cfigure\u003e\u003cimg src=\"/img/river-6068374_640.jpg\" alt=\"blocks\" class=\"mx-auto my-0 rounded-md\" /\u003e\n\u003c/figure\u003e\n\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eImage by Benjamin Zocholl from Pixabay\u003c/em\u003e\u003c/p\u003e","title":"Introduction to Apache Kafka"},{"content":"The amount of data being generated in the world is staggering. An estimated 79 zettabytes [1] of data will be generated in 2025, and that number is expected to grow in the coming years. The data can come from a variety of sources, including sensors, video processing, applications, and social media [2]. This is where Event Stream processing can help analyze data at the source, or in transit, to analyze, verify, transform, enrich, and or validate the data.\nImage by Jörg Vieli from Pixabay\nTraditionally, data has been processed in batches, with data collection over a period of time and then processing all at once. However, this approach is not ideal for large volumes of data that are in a constant state of flux, where not all data is relevant or complete, or where data can be out of date very quickly. Event streaming processing is an approach to data processing that is designed to handle large amounts of data that are constantly changing. [3]\nWith event streaming processing, businesses can make decisions and take action in real-time. This can give them a competitive advantage by allowing them to react faster to changing markets. It can help businesses improve their efficiency. By processing data in real-time, businesses can avoid storing large amounts of data. This can save companies money and space, improve their customer experience, and provide personalized experiences to their customers.\nCommon use cases for streaming processing include Fraud detection: Event streaming processing can be used to detect fraud in real-time. For example, a bank could use event streaming processing to track transactions and identify suspicious activity. Customer Behavior Analysis: Event streaming processing can be used to analyze customer behavior in real-time. For example, a retailer could use event streaming processing to track customer foot traffic and identify trends. IoT applications: IoT applications are well suited to event-streaming processing. For example, a smart city could use event streaming processing to collect data from sensors and take action in real-time.\nNowadays it is still a challenge to develop, operate and integrate stream processing systems into the business workflows. Monitoring, fine tuning for cost efficiency, handling backpressure and fault tolerance among many of the challenges are still part of the day-to-day work for developers and operators even when selecting cloud solutions.\nEvent Streaming Processing stands as a critical technology in the digital age, driving real-time communication and processing. As we\u0026rsquo;ve seen, its potential is limitless, enabling organizations to streamline processes, respond quickly, and improve user experiences. By embracing Event Streaming, organizations can have real-time insights and continue to innovate in an ever-evolving digital landscape.\nReferences #[1] Internet of Things and data placement\n[2] State of IoT 2023: Number of connected IoT devices growing 16% to 16.7 billion globally\n[3] Event Stream Processing\n","date":"August 7, 2023","permalink":"/posts/event_stream_processing/","section":"Posts","summary":"\u003cp\u003eThe amount of data being generated in the world is staggering. An estimated 79\nzettabytes \u003ca href=\"https://infohub.delltechnologies.com/l/edge-to-core-and-the-internet-of-things-2/internet-of-things-and-data-placement#\" target=\"_blank\" rel=\"noreferrer\"\u003e[1]\u003c/a\u003e\nof data will be generated in 2025, and that number is expected to grow in the coming years. The data can come from a\nvariety of sources, including sensors, video processing, applications, and social\nmedia \u003ca href=\"https://iot-analytics.com/number-connected-iot-devices/\" target=\"_blank\" rel=\"noreferrer\"\u003e[2]\u003c/a\u003e. This is where \u003cstrong\u003eEvent Stream processing\u003c/strong\u003e can help\nanalyze data at the source, or in transit, to analyze, verify, transform, enrich, and or validate the data.\u003c/p\u003e","title":"What is Event Stream Processing?"},{"content":"With the IoT revolution, software has become a key component. Developing software products that are: lightweight, efficient, and scalable requires the proper tools such as MQTT and C++.\nThis post describes how to install libraries, configure the CMake and build a simple C++ client that can post a literal text message to a cloud MQTT broker.\nWhy? #So what is MQTT, and why choose C++?\nMQTT [1] is a publishing/subscribe messaging protocol that is lightweight and efficient, scalable, reliable, secure, and well-supported [2], and ideal for connecting remote devices with a small code footprint and minimal network bandwidth.\nC++ is a general-purpose programming, which is the third most popular language[3] among the most energy-efficient programs [4] with a large set of libraries from the standard library and Boost and wide hardware support for high performance such as the Cuda library or the Intel Library\nFor further information regarding MQTT check the MQTT essentials\nTutorial #For this tutorial, we use C++ 17, CMake, and the Paho MQTT CPP Client. https://ports.macports.org/port/paho.mqtt.cpp/. Is assumed some C++ and CMake experience. The complete code can be found at https://gitlab.com/hurtadosanti/MqttClient.\nInstall the Libraries #This requires three steps on an ubuntu 22.04 machine: 1) Install the build libraries, 2) Install Paho for C and 3) Install Paho for C++.\nNOTE: In a Mac, MacPorts provides the Paho MQTT CPP packet\nInstall the compilation essentials\nsudo apt install build-essential cmake git pkg-config gcc g++ libssl-dev Clone build and install the Eclipse Paho C Client Library for the MQTT Protocol.\ngit clone https://github.com/eclipse/paho.mqtt.c.git cd paho.mqtt.c cmake -Bbuild -H. -DPAHO_ENABLE_TESTING=OFF -DPAHO_BUILD_STATIC=ON -DPAHO_WITH_SSL=ON -DPAHO_HIGH_PERFORMANCE=ON sudo cmake --build build/ --target install sudo ldconfig Clone, build and install the Eclipse Paho MQTT C++ Client Library.\ncd ~/software git clone https://github.com/eclipse/paho.mqtt.cpp cd paho.mqtt.cpp cmake -Bbuild -H. -DPAHO_BUILD_STATIC=ON -DPAHO_BUILD_DOCUMENTATION=FALSE -DPAHO_BUILD_SAMPLES=FALSE sudo cmake --build build/ --target install sudo ldconfig Configure CMake #The required Libraries are Threads and the PahoMqtt.\nset(CMAKE_CXX_STANDARD 17) find_package(Threads REQUIRED) find_package(PahoMqttCpp REQUIRED) Create a HiveMQ Cloud account #Create and get the credentials from https://console.hivemq.cloud\nWrite the Client #To write the client we use the Paho Async Client.\nFirst, we write the callback to handle the messages\nclass LiteralCallback : public virtual mqtt::callback, public virtual mqtt::iaction_listener { private: mqtt::async_client \u0026amp;client; const std::string topic; public: LiteralCallback(mqtt::async_client \u0026amp;client, std::string \u0026amp;topic) : client(client), topic(topic) { } void connected(const mqtt::string \u0026amp;string) override { callback::connected(string); mqtt::subscribe_options options; mqtt::properties properties; client.subscribe(topic, 0, options, properties); std::cout \u0026lt;\u0026lt; \u0026#34;Subscribed to topic\u0026#34;\u0026lt;\u0026lt;topic\u0026lt;\u0026lt; std::endl; }; void message_arrived(mqtt::const_message_ptr message) override { callback::message_arrived(message); std::cout \u0026lt;\u0026lt; \u0026#34;message arrived on topic: \u0026#34; \u0026lt;\u0026lt; message-\u0026gt;get_topic() \u0026lt;\u0026lt; \u0026#34;, and message: \u0026#34; \u0026lt;\u0026lt; message-\u0026gt;get_payload() \u0026lt;\u0026lt; std::endl; }; void on_failure(const mqtt::token \u0026amp;asyncActionToken) override { std::cout \u0026lt;\u0026lt; \u0026#34;on failure\u0026#34; \u0026lt;\u0026lt; std::endl; if (asyncActionToken.get_message_id() != 0) std::cout \u0026lt;\u0026lt; \u0026#34; for token: [\u0026#34; \u0026lt;\u0026lt; asyncActionToken.get_message_id() \u0026lt;\u0026lt; \u0026#34;]\u0026#34; \u0026lt;\u0026lt; std::endl; std::cout \u0026lt;\u0026lt; std::endl; } void on_success(const mqtt::token \u0026amp;asyncActionToken) override { std::cout \u0026lt;\u0026lt; \u0026#34;on success\u0026#34; \u0026lt;\u0026lt; std::endl; } ~LiteralCallback() override = default; }; Afterward, write the client and pass the cloud credentials through program arguments.\nint main(int argc, char *argv[]) { const std::string clientId(\u0026#34;cloud_client_2\u0026#34;); std::string topicName(\u0026#34;test\u0026#34;); const int qos = 1; std::string username; std::string password; std::string serverAddress; if (argc \u0026gt; 3) { username = argv[1]; password = argv[2]; serverAddress = argv[3]; } else { std::cerr \u0026lt;\u0026lt; \u0026#34;Broker Settings should be set\u0026#34; \u0026lt;\u0026lt; std::endl; return 1; } try { mqtt::async_client cli(serverAddress, clientId); mqtt::connect_options connectOptions(username, password); mqtt::ssl_options sslOptions; connectOptions.set_ssl(sslOptions); LiteralCallback cb(cli, topicName); cli.set_callback(cb); auto connection = cli.connect(connectOptions, nullptr, cb); connection-\u0026gt;wait(); mqtt::topic topic(cli, topicName, qos); auto publishToken = topic.publish(\u0026#34;Cloud Client Testing\u0026#34;); std::cout\u0026lt;\u0026lt;\u0026#34;Message send\u0026#34;\u0026lt;\u0026lt;std::endl; publishToken-\u0026gt;wait(); std::cout\u0026lt;\u0026lt;\u0026#34;Message received\u0026#34;\u0026lt;\u0026lt;std::endl; cli.disconnect()-\u0026gt;wait(); } catch (const mqtt::exception \u0026amp;exc) { std::cerr \u0026lt;\u0026lt; exc \u0026lt;\u0026lt; std::endl; return 2; } return 0; } Compile and Build #To compile, create a build folder for the binaries and run the cmake task.\nmkdir build \u0026amp;\u0026amp; cd build cmake ../ When successful build the binary using make.\nmake Run a test #Finally, execute the binary mqtt_client passing the three program arguments in the correct order.\nmqtt_client \u0026lt;username\u0026gt; \u0026lt;password\u0026gt; ssl://\u0026lt;url\u0026gt; Now you should see the following:\non success on success Subscribed to topictest Message send message arrived on topic: test, and message: Cloud Client Testing Message received References\n[1] MQTT: The Standard for IoT Messaging [2] What Is Mqtt? [3] TIOBE Index [4] Which Programming Languages Use the Least Electricity? ","date":"January 8, 2023","permalink":"/posts/mqtt_cpp_client/","section":"Posts","summary":"\u003cp\u003eWith the IoT revolution, software has become a key component. Developing software products that are: lightweight,\nefficient,\nand scalable requires the proper tools such as MQTT and C++.\u003c/p\u003e\n\u003cp\u003eThis post describes how to install libraries, configure the CMake and build a simple C++ client that can post a literal\ntext message to a cloud MQTT broker.\u003c/p\u003e\n\u003ch2 id=\"why\" class=\"relative group\"\u003eWhy? \u003cspan class=\"absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100\"\u003e\u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700\" style=\"text-decoration-line: none !important;\" href=\"#why\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\u003c/span\u003e\u003c/h2\u003e\u003cp\u003eSo what is MQTT, and why choose C++?\u003c/p\u003e","title":"MQTT Client using Paho cpp"},{"content":"In today\u0026rsquo;s world, even coffee machines send data online. However, this isn\u0026rsquo;t a new concept. The Internet of Things (IoT) has been around since 1999 when Procter \u0026amp; Gamble proposed RFID devices to track products. By 2011, we saw the emergence of applications like Google Nest smart thermostats [1].\nPhoto by Leio McLaren from Unsplash\nFast-forward to 2022, and we now have a multitude of solutions that send data to the cloud. These solutions span across homes, vehicles, and industries:\nIn homes, we have gadgets like camera doorbells that keep us informed even when we\u0026rsquo;re away. In industries, there are measurement and control devices that make intelligent decisions. Vehicles are equipped with video processing systems for event handling and over-the-air updates. However, implementing these solutions on a large scale introduces complexity. Imagine a doorbell relying on a constant internet connection or a machine making critical decisions based on contextual information. These devices must operate independently for extended periods, receive software updates, and ensure data integrity without constant supervision.\nThe Power of Edge Computing #The answer lies in making these devices smarter and less dependent on the internet, a concept known as Edge Computing. Edge Computing involves moving data center processing and services closer to data sources [2][3]. Its goals include reducing response times, bandwidth usage, and data loss, while also enhancing reliability for real-time decision-making.\nWhat Sets Edge Computing Apart? #In essence, Edge Computing extends cloud capabilities to IT administrators while minimizing the associated costs, especially when dealing with millions of devices.\nSolution Providers #Various solution providers simplify the deployment and management of fleets of edge devices. A few noteworthy options include:\nCisco IoT Edge software and hardware [2], offering comprehensive stack solutions. Balena [4], which allows flexibility in choosing hardware and offers a \u0026ldquo;pay as you go\u0026rdquo; solution with customizable components. Utilizing a fully open-source solution with Yocto [5] Linux for in-house development. What\u0026rsquo;s on the Horizon? #In a future post, we will delve into the technical challenges to consider when designing and implementing impactful solutions without compromising on quality.\nReferences # [1] IoT History [2] Cisco Edge Computing [3] Red Hat Edge Computing [4] Balena [5] Yocto Project ","date":"December 3, 2022","permalink":"/posts/advent_edge_computing/","section":"Posts","summary":"\u003cp\u003eIn today\u0026rsquo;s world, even coffee machines send data online. However, this isn\u0026rsquo;t a new concept. The Internet of Things (IoT)\nhas been around since 1999 when Procter \u0026amp; Gamble proposed RFID devices to track products. By 2011, we saw the emergence\nof applications like Google Nest smart\nthermostats \u003ca href=\"https://www.sciencedirect.com/science/article/abs/pii/S0164121217300018\" target=\"_blank\" rel=\"noreferrer\"\u003e[1]\u003c/a\u003e.\u003c/p\u003e\n\u003cp\u003e\n\n\n\n\n\n\n  \n  \n\u003cfigure\u003e\u003cimg src=\"/img/edge.jpg\" alt=\"Edge Computing\" class=\"mx-auto my-0 rounded-md\" /\u003e\n\u003c/figure\u003e\n\u003c/p\u003e\n\u003cp\u003e\u003cem\u003ePhoto by Leio McLaren from Unsplash\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eFast-forward to 2022, and we now have a multitude of solutions that send data to the cloud. These solutions span across\nhomes, vehicles, and industries:\u003c/p\u003e","title":"The Advent of Edge Computing"},{"content":"Introduction #Developing software for embedded devices has many challenges. One of them is to be able to build software for different platforms than the ones used by the developers. It is necessary to have a clear set of dependencies and reduce the overhead of optional libraries.\nIn this post, I will present briefly how Ubuntu 20.04 can cross-compile a C++ executable or use Docker to have a container running Arm64 on an Amd64 machine to build a package for Ubuntu 20.04.\nPhoto by Ketut Subiyanto from Pexels\nCross-compilation #Cross-compile allows a developer to create an application for a different computer architecture than the one used for development and to have enough resources not only to run, but also to benchmark and debug the software faster.\n\u0026ldquo;To cross-compile is to build on one platform a binary that will run on another platform\u0026rdquo;[1]\nAlternatives #Today there are different tools that help with cross-compilation, to name a few:\nThe Yocto project that can generate not only binaries but, complete Linux distributions[2] Crosstool-NG that help\u0026rsquo;s to build only toolchains[3] Docker together with Qemu could help build the binaries for arm64 and amd64 machines. Cross-tool packages from a Linux distribution. On this post we focus only on the last two that works for us developing for Raspberry Pi\u0026rsquo;s.\nRun a Docker Arm64 container on a Amd64 machine #This alternative is quite simple to run and is very well supported by debian distributions.\nUsing ubuntu 20.04 try the following command\ndocker run arm64v8/ubuntu See that we have to prepend the architecture arm64v8:\nYou should see the message:\nWARNING: The requested image\u0026#39;s platform (linux/arm64/v8) does not match the detected host platform (linux/amd64) and no specific platform was requested Install the library on ubuntu sudo apt install qemu-user-static and run the command:\ndocker run --platform linux/arm64 arm64v8/ubuntu uname -m You should see the output:\naarch64 Congratulations, now you are running ubuntu on an amd64 system, you can also build images with docker build. It can be not as efficient as runing directly on the machine, but will allow you to build and run packages for Arm64 or 32 without much more effort.\nUsing the cross-tool packages for GCC #Another option is to build using GCC, this will work for a program without dependencies but will make it very difficult to have all the dependencies for the right architecture.\nDependencies\nsudo apt install build-essential autoconf libtool cmake pkg-config git Install the libraries for cross-compilation\nsudo apt install gcc-aarch64-linux-gnu g++-aarch64-linux-gnu g++-arm-linux-gnueabihf Prepare the CMAKE file with the following dependencies\ncmake_minimum_required(VERSION 3.1) project(test) set(CMAKE_SYSTEM_NAME Linux) set(CMAKE_SYSTEM_PROCESSOR aarch64) set(CMAKE_CXX_STANDARD 17) SET(CMAKE_C_COMPILER aarch64-linux-gnu-gcc) SET(CMAKE_CXX_COMPILER aarch64-linux-gnu-g++) set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER) set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY) set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY) set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY) add_executable(${PROJECT_NAME} main.cpp) The C++ code that will be executed\n//main.cpp #include \u0026lt;iostream\u0026gt; int main() { std::cout \u0026lt;\u0026lt; \u0026#34;Cross compiled\u0026#34; \u0026lt;\u0026lt; std::endl; return 0; } Build\nmkdir build \u0026amp;\u0026amp; cd build \u0026amp;\u0026amp; cmake .. \u0026amp;\u0026amp; make Check the architecture for the binary with file test the output will be test: ELF 64-bit LSB shared object, ARM aarch64 ...\nRun the binary on an arm64 machine\ndocker run -it --rm --platform linux/arm64 -v $PWD:/test arm64v8/ubuntu /test/test Congrats you should see: Cross compiled on the standard out.\nHaving a program today that runs on different processor architectures is a given. For example, Java or Python allows developers to run applications almost transparently on different computer architectures. Still, the capability of doing so on systems with low resources is higly beneficial to be able to test first and then cross-compile on the target device.\n","date":"February 22, 2022","permalink":"/posts/cross_compilation/","section":"Posts","summary":"\u003ch2 id=\"introduction\" class=\"relative group\"\u003eIntroduction \u003cspan class=\"absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100\"\u003e\u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700\" style=\"text-decoration-line: none !important;\" href=\"#introduction\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\u003c/span\u003e\u003c/h2\u003e\u003cp\u003eDeveloping software for embedded devices has many challenges. One of them is to be able to build software for different\nplatforms than the ones used by the developers. It is necessary to have a clear set of dependencies and reduce the\noverhead of optional libraries.\u003c/p\u003e\n\u003cp\u003eIn this post, I will present briefly how Ubuntu 20.04 can cross-compile a C++ executable or use Docker to have a\ncontainer running Arm64 on an Amd64 machine to build a package for Ubuntu 20.04.\u003c/p\u003e","title":"Linux Cross Development"},{"content":"On this project we simulate a full home service robot capable of navigating to pick up and deliver virtual objects. The robot we use is the turtlebot provided by ROS. The robot is capable to localize, map and navigate using a fusion of LIDAR, camera and odometer. We have used the SLAM_gmapping module and the AMCL adaptive Monte Carlo localization provided by ROS.\nLocalization and Mapping #GMapping is a highly efficient Rao-Blackwellized particle filer to learn grid maps from laser range data. Using SLAM_gmapping, you can create a 2-D occupancy grid map from laser and pose data collected by the robot.\nNavigation #For navigation and object avoidance we use the ROS Navigation stack, which is based on the Dijkstra\u0026rsquo;s, a variant of the Uniform Cost Search algorithm.\nDependencies # Ubuntu 16.04 ROS Kinetic Kame AMCL ROS Module slam_gmapping References # https://github.com/hurtadosanti/home-service-robot ","date":"March 31, 2021","permalink":"/posts/service-robot/","section":"Posts","summary":"\u003cp\u003eOn this \u003ca href=\"https://github.com/hurtadosanti/home-service-robot\" target=\"_blank\" rel=\"noreferrer\"\u003eproject\u003c/a\u003e we simulate a full home service robot capable of\nnavigating to pick up and deliver virtual objects. The robot we use is the turtlebot provided by ROS. The robot is\ncapable to localize, map and navigate using a fusion of LIDAR, camera and odometer. We have used the SLAM_gmapping\nmodule and the AMCL adaptive Monte Carlo localization provided by ROS.\u003c/p\u003e\n\u003chr\u003e\n\u003ch2 id=\"localization-and-mapping\" class=\"relative group\"\u003eLocalization and Mapping \u003cspan class=\"absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100\"\u003e\u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700\" style=\"text-decoration-line: none !important;\" href=\"#localization-and-mapping\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\u003c/span\u003e\u003c/h2\u003e\u003cp\u003eGMapping is a highly efficient Rao-Blackwellized particle filer to learn grid maps from laser range data. Using\nSLAM_gmapping, you can create a 2-D occupancy grid map from laser and pose data collected by the robot.\u003c/p\u003e","title":"Service Robot"},{"content":"The Robot Operating System or ROS is a framework and an open-source project. It is a project that is currently being developed and used by many industries around the world. Together, with applications that include Self-Driving Vehicles https://www.autoware.auto/ and other industrial robots.\nPhoto by Dominik Scythe on Unsplash\nFor me as a software engineer has been interesting to have the ability to design a simple robot and have it moving around different environments with ease. After having finished the Self-Driving Nanodegree of Udacity I continue my path with the Robotics Nanodegree. In very little time, I have created a robot and its world environment and have the robot chase a white ball. I have also learned different localization algorithms different than the Extended Kalman Filter I have developed before. Using the adaptive Monte Carlo localization (ACML) ROS package was much simpler on the latest localization project I have finished letting the robot find where it is.\nI believe ROS is a project that presents many of the benefits of open source, where many people can contribute packages and benefit from a framework when starting new projects or improving current ones. The field of robotics is a broad one, where thousands of disciplines need to integrate into one or several solutions.\n","date":"January 30, 2021","permalink":"/posts/ros_introduction/","section":"Posts","summary":"\u003cp\u003eThe \u003ca href=\"https://www.ros.org/\" target=\"_blank\" rel=\"noreferrer\"\u003eRobot Operating System\u003c/a\u003e or ROS is a framework and an open-source project. It is a project\nthat is currently being developed and used by many industries around the world. Together, with applications that include\nSelf-Driving Vehicles \u003ca href=\"https://www.autoware.auto/\" target=\"_blank\" rel=\"noreferrer\"\u003ehttps://www.autoware.auto/\u003c/a\u003e and other industrial robots.\u003c/p\u003e\n\u003cp\u003e\n\n\n\n\n\n\n  \n  \n\u003cfigure\u003e\u003cimg src=\"/img/robot.jpg\" alt=\"robot\" class=\"mx-auto my-0 rounded-md\" /\u003e\n\u003c/figure\u003e\n\u003c/p\u003e\n\u003cp\u003e\u003cem\u003ePhoto\nby \u003ca href=\"https://unsplash.com/@drscythe?utm_source=unsplash\u0026amp;amp;utm_medium=referral\u0026amp;amp;utm_content=creditCopyText\" target=\"_blank\" rel=\"noreferrer\"\u003eDominik Scythe\u003c/a\u003e\non \u003ca href=\"https://unsplash.com/s/photos/robot?utm_source=unsplash\u0026amp;amp;utm_medium=referral\u0026amp;amp;utm_content=creditCopyText\" target=\"_blank\" rel=\"noreferrer\"\u003eUnsplash\u003c/a\u003e\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eFor me as a software engineer has been interesting to have the ability to design a simple robot and have it moving\naround different environments with ease. After having finished\nthe \u003ca href=\"https://www.udacity.com/course/self-driving-car-engineer-nanodegree--nd013\" target=\"_blank\" rel=\"noreferrer\"\u003eSelf-Driving Nanodegree\u003c/a\u003e of Udacity I\ncontinue my path with the Robotics Nanodegree. In very little time, I have created a robot and its world environment and\nhave the robot \u003ca href=\"https://github.com/hurtadosanti/go_chase_it\" target=\"_blank\" rel=\"noreferrer\"\u003echase a white ball\u003c/a\u003e. I have also learned different\nlocalization algorithms different than the \u003ca href=\"https://advt3.com/posts/robot_localization/\" target=\"_blank\" rel=\"noreferrer\"\u003eExtended Kalman Filter\u003c/a\u003e I have\ndeveloped before. Using the adaptive Monte Carlo localization (ACML) ROS package was much simpler on\nthe \u003ca href=\"https://github.com/hurtadosanti/localization\" target=\"_blank\" rel=\"noreferrer\"\u003elatest localization project\u003c/a\u003e I have finished letting the robot find\nwhere it is.\u003c/p\u003e","title":"The Robot Operating System (ROS) introduction"},{"content":" A proportional–integral–derivative (PID) controller is a commonly used algorithm on control loops. The controller takes a signal input and calculates the desired actuator output. The PID includes in the feedback loop the present error (P), past error memory (I), and future (D) to reduce overshooting, and base on these values apply correction. On this project we have implemented a PID controller in C++ to maneuver the vehicle around the simulator track.\nPhoto by Florin Kozma on Unsplash\nThe implementation of the algorithm is fairly straightforward. Every time we get an error, we stored the variables and compute a new value. We use two PID controllers one for steering and one for throttle control, each with its initial constants. Given the fact that in practice the integral error can get saturated we clamped the saturation to a max value, which provides us with smoother correction when the car started oscillating too much.\nThe Video result shows how the car goes around the track.\nAlthough many algorithms can do the parameter adjustment, we choose to understand them manually. Tuning each parameter at a time by hand helps understand how each one of them works. Although throttle control uses PID, fixing parameters is not as evident as steering control.\nThe parameters correspond as follows for steering.\nP constant value was setup to 0.05. The value defines how much correction is done. If the value is set to say 1, the car will continuously correct the direction making it oscillate heavily on a straight line.\nI Constant value was set 0.001 with a max value of 200. This value was set to 0 initially, and when the other two values P and D were set, with put a very small value that helps the car driver a bit smoother.\nD constant value was set to 3. Finally, the D value was set high since it helps to turn on the curves harder.\nThe parameters correspond as follows for throttle.\nP constant value was setup to 0.09\nI constant value was set 0.0001 with a max value of 500\nD constant value was set to 0.5\nReferences #https://www.ni.com/en-us/innovations/white-papers/06/pid-theory-explained.html\nhttps://github.com/udacity/CarND-Controls-PID\nhttps://github.com/udacity/self-driving-car-sim/releases\nhttps://review.udacity.com/#!/rubrics/1972/view\n","date":"December 6, 2020","permalink":"/posts/pid_controller/","section":"Posts","summary":"\u003cp\u003e\u003ca href=\"http://www.udacity.com/drive\" target=\"_blank\" rel=\"noreferrer\"\u003e\n\n\n\n\n\n\n  \n  \n\u003cfigure\u003e\u003cimg src=\"https://s3.amazonaws.com/udacity-sdc/github/shield-carnd.svg\" alt=\"Udacity - Self-Driving Car NanoDegree\" class=\"mx-auto my-0 rounded-md\" /\u003e\n\u003c/figure\u003e\n\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eA proportional–integral–derivative \u003cstrong\u003e(PID)\u003c/strong\u003e controller is a commonly used algorithm on control loops. The controller\ntakes a signal input and calculates the desired actuator output. The PID includes in the feedback loop the present\nerror (P), past error memory (I), and future (D) to reduce overshooting, and base on these values apply correction. On\nthis \u003cstrong\u003e\u003ca href=\"https://github.com/hurtadosanti/PIDController\" target=\"_blank\" rel=\"noreferrer\"\u003eproject\u003c/a\u003e\u003c/strong\u003e we have implemented a PID controller in C++ to\nmaneuver the vehicle around the simulator track.\u003c/p\u003e","title":"PID Controller"},{"content":" One of the primary purposes of a self-driving vehicle is to be able to arrive at a target location without human intervention. To accomplish this task, the vehicle must be able to figure out a route to arrive at the destination position (Path Planning) and must be able to navigate the obstacles (Motion Planning) to arrive safely at the intended location. For example, if the car wants to arrive at your home, it should first use the GPS and find the shortest route. From this time has been calculated it is required to get context information such as lanes and other vehicles to navigate safely to your house [1]\nImage by Free-Photos from Pixabay\nIn this project[2], we have implemented a planner that allows the simulated vehicle to navigate through three lanes of a highway, avoiding: Collide with other cars, exceed speed, acceleration, and/or jerk, and maintaining the lane until is safe to maneuver.\nChange Lanes #The planner::change_lanes method uses a state machine to decide what maneuver must be executed. Using the Frenet coordinate system the vehicle calculates a safe path to drive through traffic.\nUses sensor fusion to identify cars that are on the same lane, if a vehicle is detected on the same lane\nWhen a car is detected first checks that are possible to use the left lane to the overpass. It checks that there are no cars in front or behind within a safe distance(including acceleration)\nAfter checks that is possible to use the right lane to the overpass. It checks that there are no cars in front or behind within a safe distance(including acceleration)\nFinally, if a car is detected to be too close, the vehicle slows down, and when it is possible turns to the left or right lanes\nWaypoint Planner #The planner::waypoint_planner method verify when is safe to drive using the change_lanes method and drives using the map coordinates.\nGets the car position and map waypoints in S, X, Y\nCalculates the yaw\nConvert the frenet points into map coordinates\nCalculating with the actual lane, the next points required to navigate\nTransform to vehicle coordinates\nSmooths the navigation, yaw, and path using the spline library\nFinally converts back to the X, and Y coordinates to be pass to the vehicle control system\nConclusion #Throughout this project, we apply the knowledge gathered through multiple lessons and partial implementations of algorithms for breadth/depth-first search, Hybrid A*, naive algorithms, cost functions, and Jerk minimizing trajectories which are some of the basis of modern self-driving vehicles. However, much remains to be done to achieve a complete self-autonomy level 5.\nAs with many of the projects before, I have enjoyed implementing the algorithms and understanding their part in the bigger picture.\nReferences #[1] A Survey of Motion Planning and ControlTechniques for Self-driving Urban Vehicles https://arxiv.org/pdf/1604.07446.pdf\n[2] Highway Path Planning Implementation https://github.com/hurtadosanti/HighwayPathPlanning\n","date":"November 30, 2020","permalink":"/posts/path_motion_planning/","section":"Posts","summary":"\u003cp\u003e\u003ca href=\"http://www.udacity.com/drive\" target=\"_blank\" rel=\"noreferrer\"\u003e\n\n\n\n\n\n\n  \n  \n\u003cfigure\u003e\u003cimg src=\"https://s3.amazonaws.com/udacity-sdc/github/shield-carnd.svg\" alt=\"Udacity - Self-Driving Car NanoDegree\" class=\"mx-auto my-0 rounded-md\" /\u003e\n\u003c/figure\u003e\n\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eOne of the primary purposes of a self-driving vehicle is to be able to arrive at a target location without human\nintervention. To accomplish this task, the vehicle must be able to figure out a route to arrive at the destination\nposition (Path Planning) and must be able to navigate the obstacles (Motion Planning) to arrive safely at the intended\nlocation. For example, if the car wants to arrive at your home, it should first use the GPS and find the shortest route.\nFrom this time has been calculated it is required to get context information such as lanes and other vehicles to\nnavigate safely to your house \u003ca href=\"https://arxiv.org/pdf/1604.07446.pdf\" target=\"_blank\" rel=\"noreferrer\"\u003e[1]\u003c/a\u003e\u003c/p\u003e","title":"Path and Motion Planning for autonomous vehicles"},{"content":"Establishing a particular position in the world (Localization) is a problem solved by Global position systems (GPS). However, GPS accuracy for autonomous vehicles does not provide the right information at the right time. The accuracy and response time are not sufficiently precise, the distance is given in meters not centimeters of error, as well as environmental and contextual conditions such as rain, clouds, or nearby buildings increase the noise[1]. Navigation for mobile robots requires to solve four challenges: Environmental perception, localization, path planning(cognition), and vehicle or motion control[2].\nImage by Tomislav Jakupec from Pixabay\nSolution #On this project[3], we have implemented the Particle Filter to solve the localization problem.\nThere are different approaches to solve these issues, based on Lidar, Radar, and/or Video, that support in providing the right accuracy. The drawback is that they require more computing resources on the vehicle. Most know methods use Bayesian filters, such as Kalman or Particle Filters[1].\nAlgorithm #The particle filter algorithm occurs in four (4) steps:\nInitialize all the particles using GPS information, considering the noise Given the yaw rate, time change, and velocity predict the position of all particles Provided the observed points and the map update the belief(weight) of all the particles with the landmarks close to our actual position Finally, resample the weight distribution of the particles In the following [4], you can find a deep explanation of the algorithm.\nImplementation #Our implementation[3] receives a map of the vehicle information from the simulator and provides the best particle belief found. It is also supplied the ground truth to calculate the RMSE to be aware of the error from the calculations.\nGiving the math and probability calculation explanation, we did implement the code to find a solution that worked on the simulator. It is a very good exercise to have implemented in C++ to understand some of the peculiarities that one has when developing such solutions.\nReferences #[1] Robotic Autonomy course Carnegie Mellon, Chapter 5 http://www.cs.cmu.edu/~rasc/Download/AMRobots5.pdf\n[2] A Particle Filter Localization Method Using 2D Laser Sensor Measurements and Road Features for Autonomous Vehicle http://downloads.hindawi.com/journals/jat/2019/3680181.pdf\n[3] Particle Filter for localization Implementation https://github.com/hurtadosanti/ParticleFilterLocalization\n[4] Particle Filters Video https://www.youtube.com/watch?v=lzN18y_z6HQ\n","date":"November 13, 2020","permalink":"/posts/robot_localization/","section":"Posts","summary":"\u003cp\u003eEstablishing a particular position in the world (Localization) is a problem solved by Global position systems (GPS).\nHowever, GPS accuracy for autonomous vehicles does not provide the right information at the right time. The accuracy and\nresponse time are not sufficiently precise, the distance is given in meters not centimeters of error, as well as\nenvironmental and contextual conditions such as rain, clouds, or nearby buildings increase the\nnoise\u003ca href=\"http://www.cs.cmu.edu/~rasc/Download/AMRobots5.pdf\" target=\"_blank\" rel=\"noreferrer\"\u003e[1]\u003c/a\u003e. Navigation for mobile robots requires to solve four\nchallenges: Environmental perception, localization, path planning(cognition), and vehicle or motion\ncontrol\u003ca href=\"http://downloads.hindawi.com/journals/jat/2019/3680181.pdf\" target=\"_blank\" rel=\"noreferrer\"\u003e[2]\u003c/a\u003e.\u003c/p\u003e","title":"Robot Localization"},{"content":"Object tracking has several applications, not only in the military but also today in autonomous vehicles, for example, to avoid collisions. Autonomous vehicles use different sensors to understand their context, cameras, radars and/or lasers. Sensor fusion merges multiple inputs and considers the strengths of each of the them. To help tracking better the different objects using Kallman Filters [1].\nAn algorithm used for sensor fusion is the Kalman filter[2], which is a recursive algorithm that helps solve the problem of tracking moving objects. Each iteration of the algorithm consists of two steps the update and measurement of the object position and velocity. The benefit is that it minimizes the root mean square error (RMSE) on each iteration.\nPhoto by Samuele Errico Piccarini on Unsplash\nHowever, the Kalman filter can only be used on linear problems, such as the usage of Lidar sensors, to calculate the position of the tracked object. On the other hand, the Extended Kalman Filter can track objects with a non-linear sensor such as the Radar of a car, that can help calculate the velocity of the tracked object[3].\nIn this project[4], we have implemented in C++ both algorithms and using the Udacity Simulator[5] to verify the implementation results.\nImplementing the filter in C++ over Python increases the complexity of the matrix calculations, but also increases the performance. Debugging using the Eigen library can be a daunting task, However, having implemented before on python helped to have a clear understanding of the algorithms and the expectations.\nUsing a docker image for the compilation together with Clion also makes the implementation and debugging faster and easier.\nAs always is an interesting project with a direct application and visualization of the solution.\nReferences #[1] Understanding Kalman Filters https://www.mathworks.com/videos/series/understanding-kalman-filters.html\n[2] Moving Object Tracking Using Kalman Filter https://doi.org/10.1109/ICACCT.2018.8529402\n[3] Extended Kalman filter for extended object tracking https://doi.org/10.1109/ICASSP.2017.7952985\n[4] Extended Kalman filter implementation https://github.com/hurtadosanti/ExtendedKalmanFilter\n[5] Udacity Simulator https://github.com/udacity/self-driving-car-sim/releases/\n","date":"October 25, 2020","permalink":"/posts/object_tracking/","section":"Posts","summary":"\u003cp\u003eObject tracking has several applications, not only in the military but also today in autonomous vehicles, for example,\nto avoid collisions. Autonomous vehicles use different sensors to understand their context, cameras, radars and/or\nlasers. Sensor fusion merges multiple inputs and considers the strengths of each of the them. To help tracking better\nthe different objects using Kallman\nFilters \u003ca href=\"https://www.mathworks.com/videos/series/understanding-kalman-filters.html\" target=\"_blank\" rel=\"noreferrer\"\u003e[1]\u003c/a\u003e.\u003c/p\u003e\n\u003cp\u003e\u003ca href=\"http://www.udacity.com/drive\" target=\"_blank\" rel=\"noreferrer\"\u003e\n\n\n\n\n\n\n  \n  \n\u003cfigure\u003e\u003cimg src=\"https://s3.amazonaws.com/udacity-sdc/github/shield-carnd.svg\" alt=\"Udacity - Self-Driving Car NanoDegree\" class=\"mx-auto my-0 rounded-md\" /\u003e\n\u003c/figure\u003e\n\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eAn algorithm used for sensor fusion is the \u003cstrong\u003eKalman filter\u003c/strong\u003e\u003ca href=\"https://doi.org/10.1109/ICACCT.2018.8529402\" target=\"_blank\" rel=\"noreferrer\"\u003e[2]\u003c/a\u003e, which is\na recursive algorithm that helps solve the problem of tracking moving objects. Each iteration of the algorithm consists\nof two steps the update and measurement of the object position and velocity. The benefit is that it minimizes the root\nmean square error (RMSE) on each iteration.\u003c/p\u003e","title":"Object Tracking"},{"content":" Behavioral cloning is a method of capturing and replicating human skills in a computer program. In this Udacity project, we learn to use deep neural networks together with convolutional neural networks. Using the simulator, we gather the necessary information to train a neural network. Finally, we used the model together with the weights to infer the steering angle required for driving autonomously. The approach was researched and developed by Nvidia on the PiloNet system and described on the Self-Driving Car Nano Degree.\nThe steps for this project were the following:\nUse the simulator to collect data on good driving behavior Design, train and validate a model that predicts a steering angle from image data using TensorFlow 2 and Keras Use the model to drive the vehicle autonomously around the first track in the simulator. The vehicle should remain on the road for an entire loop around the track The project description can be found in the github repository. It includes the Report summarizing the results, a Jupyter notebook describing the steps to load, preprocess and train the model, and a Video for an autonomous drive example.\nIt is very satisfactory to be able to develop the full cycle of an AI system and see the results at the end of the process. Each training cycle takes time from having a hypothesis before we were able to test it and then improve it. The most challenging step was to be able to work with data that does not fit into memory, a challenge that was tackled by using a generator that loads the data in batches.\nAlthough it is impressive to drive around the track, this is just part of various complex tasks. We have not dealt yet with the question of identifying other items such as automobiles and people, to be able to make smart decisions.\n","date":"October 10, 2020","permalink":"/posts/behavioral_cloning/","section":"Posts","summary":"\u003cp\u003e\u003ca href=\"http://www.udacity.com/drive\" target=\"_blank\" rel=\"noreferrer\"\u003e\n\n\n\n\n\n\n  \n  \n\u003cfigure\u003e\u003cimg src=\"https://s3.amazonaws.com/udacity-sdc/github/shield-carnd.svg\" alt=\"Udacity - Self-Driving Car NanoDegree\" class=\"mx-auto my-0 rounded-md\" /\u003e\n\u003c/figure\u003e\n\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003e\n\n\n\n\n\n\n  \n  \n\u003cfigure\u003e\u003cimg src=\"/img/driving.jpg\" alt=\"driving\" class=\"mx-auto my-0 rounded-md\" /\u003e\n\u003c/figure\u003e\n\u003c/p\u003e\n\u003cp\u003eBehavioral cloning is a method of capturing and replicating human skills in a computer program. In this Udacity project,\nwe learn to use deep neural networks together with convolutional neural networks. Using the simulator, we gather the\nnecessary information to train a neural network. Finally, we used the model together with the weights to infer the\nsteering angle required for\ndriving \u003ca href=\"https://github.com/hurtadosanti/BehavioralCloning/blob/master/video.mp4\" target=\"_blank\" rel=\"noreferrer\"\u003eautonomously\u003c/a\u003e. The approach was\nresearched and developed by Nvidia on\nthe \u003ca href=\"https://developer.nvidia.com/blog/explaining-deep-learning-self-driving-car/\" target=\"_blank\" rel=\"noreferrer\"\u003ePiloNet system\u003c/a\u003e and described on the\nSelf-Driving Car Nano Degree.\u003c/p\u003e","title":"Behavioral Cloning"},{"content":"Building a Traffic Sign Recognition Convolutional Neural Network\nNowadays, deep neural network applications can be found everywhere, from chatbots, finance, and shopping. An interesting field of use is self-driving vehicles, where they are not only used on object recognition but also on vehicle steering. i.e Explaining How End-to-End Deep Learning Steers a Self-Driving Car. Furthermore, convolutional neural networks help using neural networks on different sets of images, most usually for object classification. A classical example dates from 1998 Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner: Gradient-Based Learning Applied to Document Recognition, Proceedings of the IEEE, 86(11):2278-2324, November 1998. Where it was demonstrated that training a machine rather than being explicitly program to do a task, using for instance classical computer vision approaches, was possible.\nWe have implemented a variation of LeNet-5 network from Pierre Sermanet and Yann LeCun: Traffic Sign Recognition with Multi-Scale Convolutional Networks, Proceedings of International Joint Conference on Neural Networks (IJCNN'11), 2011. Using TensorFlow 1.x we implemented a variation, and we accomplished an accuracy of 95%. Even though the implementation is not as robust and accurate as the one proposed, it has been interesting to learn how to use TensorFlow without the usage of Keras, which is the standard today.\nContent #The project includes detailed steps to:\nLoad the data set Explore, summarize and visualize the data set Design, train, and test a model architecture Use the model to make predictions on new images Analyze the softmax probabilities of the new images The architecture selected is base on the Lenet-5 Network with 2 convolution layers followed by a max-pooling layer and 2 fully connected layers reduced to 43 outputs, to improve the accuracy we use a dropout operation before the fully connected layers.\nThe detailed implementation of the Traffic Sign Classifier Notebook and the report describes each step accomplished.\nConclusion #Convolutional neural networks have revolutionalized not only the area of computer vision but also Neuro-linguistic programming (NLP), and time series analysis among many other fields.\nFor this project, we have begun using Tensorflow 1.x for a known simple network architecture LeNet-5. Wich requires to write several implementations details. Still, it is important for a practitioner to understand how convolutional neural networks are designed. What effects every parameter has into the results, which helps to understand in-depth the in\u0026rsquo;s and out\u0026rsquo;s.\n","date":"September 27, 2020","permalink":"/posts/traffic_sign_classifier/","section":"Posts","summary":"\u003cp\u003e\u003cstrong\u003eBuilding a Traffic Sign Recognition Convolutional Neural Network\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\n\n\n\n\n\n\n  \n  \n\u003cfigure\u003e\u003cimg src=\"/img/traffic-sign-3008267_640.jpg\" alt=\"speed limit\" class=\"mx-auto my-0 rounded-md\" /\u003e\n\u003c/figure\u003e\n\u003c/p\u003e\n\u003cp\u003e\u003ca href=\"http://www.udacity.com/drive\" target=\"_blank\" rel=\"noreferrer\"\u003e\n\n\n\n\n\n\n  \n  \n\u003cfigure\u003e\u003cimg src=\"https://s3.amazonaws.com/udacity-sdc/github/shield-carnd.svg\" alt=\"Udacity - Self-Driving Car NanoDegree\" class=\"mx-auto my-0 rounded-md\" /\u003e\n\u003c/figure\u003e\n\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eNowadays, deep neural network applications can be found everywhere, from chatbots, finance, and shopping. An interesting\nfield of use is self-driving vehicles, where they are not only used on object recognition but also on vehicle steering.\ni.e\n\u003cem\u003e\u003ca href=\"https://developer.nvidia.com/blog/explaining-deep-learning-self-driving-car/\" target=\"_blank\" rel=\"noreferrer\"\u003eExplaining How End-to-End Deep Learning Steers a Self-Driving Car\u003c/a\u003e\u003c/em\u003e.\nFurthermore, convolutional neural networks help using neural networks on different sets of images, most usually for\nobject classification. A classical example dates from 1998 \u003cem\u003eY. LeCun, L. Bottou, Y. Bengio, and P.\nHaffner:  \u003ca href=\"http://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf\" target=\"_blank\" rel=\"noreferrer\"\u003eGradient-Based Learning Applied to Document Recognition\u003c/a\u003e,\nProceedings of the IEEE, 86(11):2278-2324, November 1998.\u003c/em\u003e  Where it was demonstrated that training a machine rather\nthan being explicitly program to do a task, using for instance classical computer vision approaches, was possible.\u003c/p\u003e","title":"Traffic sign classifier"},{"content":" Self-driving vehicles will make mobility safer and faster. They for example can help increase accessibility to people that can\u0026rsquo;t drive for different reasons. Though, there are a large number of challenges than need to be tackle before can be a reality. Several applications have been implemented for Advanced Driver Assistance Systems(ADAS) to improve road safety on modern vehicles such as Forward Collision Warning, Parking Assistance, Lane Centering among many others.\nIn this article, we present a classical computer vision approach, bird’s eye view implementation for lane detection. The developed strategy is as follows:\nCompute the camera calibration matrix and distortion coefficients given a set of chessboard images\nApply a distortion correction to raw images\nUse color transforms, and gradients to create a threshold binary image\nApply a perspective transform to rectify binary image (\u0026ldquo;birds-eye view\u0026rdquo;)\nDetect lane pixels and fit to find the lane boundary\nDetermine the curvature of the lane and vehicle position with respect to the center\nWarp the detected lane boundaries back onto the original image\nOutput visual display of the lane boundaries and numerical estimation of lane curvature and vehicle position\nYou can find a thorough description here. To know what was expected to be accomplished here are the rubric points.\nComplete Video\nThe pipeline description shows how the algorithms are used, and the video pipeline processes a video stream. To see the details of the algorithms used look in the utility folder. Finally, the unit tests and integration tests are located on the tests folder.\nThe code can be found on the Github repository.\nConclusions #The approach used in this project requires several steps to process an image warp to bird view and back. Which makes it computationally expensive.\nEven though we have improved a lot since the basic approach to detect lines, there are many variables that make the algorithm unstable, on conditions such as rain or low or no lines on the street.\nAn option is to have a set of algorithms to test the external context. Also, a good option is to include sensors such as light(day/night) raining, etc to help to select the proper algorithm to be used in different contextual situations.\nOn the performance side, will be good to profile the code and see if the usage of a GPU or C++ code can improve the performance.\nThere is still a place to improve the detection of problematic zones since the sliding windows are calculated only In the first frame of the video, an option is to detect problematic polylines and try to find again the sliding windows.\nReferences #https://www.udacity.com/course/self-driving-car-engineer-nanodegree--nd013\nhttps://link.springer.com/article/10.1007/s11042-016-4184-6\n","date":"September 10, 2020","permalink":"/posts/advanced_lane_detection/","section":"Posts","summary":"\u003cp\u003e\n\n\n\n\n\n\n  \n  \n\u003cfigure\u003e\u003cimg src=\"/img/advanced-lane-detection.jpg\" alt=\"lane detection\" class=\"mx-auto my-0 rounded-md\" /\u003e\n\u003c/figure\u003e\n\u003c/p\u003e\n\u003cp\u003eSelf-driving vehicles will make mobility safer and faster. They for example can help increase accessibility to people\nthat can\u0026rsquo;t drive for different reasons. Though, there are a large number of challenges than need to be tackle before can\nbe a reality. Several applications have been implemented for Advanced Driver Assistance Systems(ADAS) to improve road\nsafety on modern vehicles such as \u003cem\u003eForward Collision Warning\u003c/em\u003e, \u003cem\u003eParking Assistance\u003c/em\u003e, \u003cem\u003eLane Centering\u003c/em\u003e among many others.\u003c/p\u003e","title":"Advanced Lane Detection"},{"content":"This is the first project for the @udacity Nano degree in self-driving car. The goals of this project is to Make a pipeline that finds lane lines on the road. The code can be found on my github repository https://github.com/hurtadosanti/FindingLaneLines.\nDescription #The pipeline is defined in four(4) steps:\nRead the image or frame Highlight sections of interest by converting the image to HSV color space and find regions in the ranges:\nlow_boundary=(0,0,200) high_boundary=(255,150,255) Select the interest region to focus only on the road and remove other possible noise\nFind the lanes lines using HoughLinesP and draw complete lines\nWhen highlighting the sections of interest the HSV color space is used which helps with not having problems between yellow and white lines, they are processed equally.\nTo get the lines I use a naive approach, of first separate lines by the positive and negative slope. Afterward, I average the slope and intercept. To find the x,y values of the lines it is used the low and high numbers on the Y-axis from the interest Region. Then, using the average slope and the average intercept, the X values are calculated.\nPotential Issues #The issues identified with the actual solution are:\nDrawing the lines on the video is noisy, an option is to maintain the old points before each iteration, so if there is no point identified the line is not jumping. For the curves in the challenge video, the slopes can be restricted to a specific range so the lines do not cross Using vectorized solutions can contribute to better performance of the algorithm At present, the region of interest is fixed, so if the horizon changes the algorithm could get confused Possible Improvements # The solution can be improved by removing the blue on the sky Using vectorized solutions can contribute to better performance of the algorithm, such as using NumPy for the calculation Probably some linear regression can help with the lines I can imagine that the car can work with simple points for the lines since the car structure is fixed and the cameras too Running the code # Start a docker instance docker run -it \u0026ndash;rm \u0026ndash;entrypoint \u0026ldquo;/run.sh\u0026rdquo; -p 8888:8888 -v $PWD:/src udacity/carnd-term1-starter-kit\nOpen the Jupyter notebook and run it, make sure you get images and videos to run the solution.\n","date":"August 27, 2020","permalink":"/posts/findinglanelines/","section":"Posts","summary":"\u003cp\u003eThis is the first project for the @udacity Nano degree in self-driving car. The goals of this project is to Make a\npipeline that finds lane lines on the road. The code can be found on my github\nrepository \u003ca href=\"https://github.com/hurtadosanti/FindingLaneLines\" target=\"_blank\" rel=\"noreferrer\"\u003ehttps://github.com/hurtadosanti/FindingLaneLines\u003c/a\u003e.\u003c/p\u003e\n\u003cp\u003e\u003ca href=\"http://www.udacity.com/drive\" target=\"_blank\" rel=\"noreferrer\"\u003e\n\n\n\n\n\n\n  \n  \n\u003cfigure\u003e\u003cimg src=\"https://s3.amazonaws.com/udacity-sdc/github/shield-carnd.svg\" alt=\"Udacity - Self-Driving Car NanoDegree\" class=\"mx-auto my-0 rounded-md\" /\u003e\n\u003c/figure\u003e\n\u003c/a\u003e\u003c/p\u003e\n\u003chr\u003e\n\u003ch3 id=\"description\" class=\"relative group\"\u003eDescription \u003cspan class=\"absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100\"\u003e\u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700\" style=\"text-decoration-line: none !important;\" href=\"#description\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\u003c/span\u003e\u003c/h3\u003e\u003cp\u003eThe pipeline is defined in four(4) steps:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003eRead the image or frame\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003e\n\n\n\n\n\n\n  \n  \n\u003cfigure\u003e\u003cimg src=\"/img/original.png\" alt=\"original\" class=\"mx-auto my-0 rounded-md\" /\u003e\n\u003c/figure\u003e\n\n\u003cbr\u003e\u003c/p\u003e","title":"Finding Lane Lines on the Road"},{"content":"Introduction #Writing an algorithm employing Cuda does not require a state of the art GPU. However, to reap the benefits of massively parallel processing it is required a powerful one. How about using a cloud instance?.\nWriten by: Santiago Hurtado\nLet\u0026rsquo;s do some quick numbers first:\nAn Intel i7-9700f processor should deliver 384 Gflops at a price of about 400 USD\nAn Nvidia Quadro P400 has a theoretical performance 641.0 GFLOPS FP32 for about 113 USD\nWhile an Nvidia GeForce RTX 2060 GPU that cost about 339 USD can produce 6.451 TFLOPS FP32.\nNow the basic Azure N6 cloud instance should include an Nvidia Tesla K80 with a theoretical performance of 4.113 TFLOPS FP32 for $0.90/hour or about $500 a month, please don\u0026rsquo;t leave it running for a month, you can buy a very good GPU for a thousand dollars.\nInstance Creation #Ok, enough of that, So how do we get an Azure VM that gives the promise of a large number of computations per second. * The setup we will be performing is only recommended for the bare usage of Cuda*. A depth tutorial can be found at the Nvidia site. Also, if you are looking for a * Data science VM* there is an easier way, follow this tutorial from Microsoft.\nMost of the time I recommend using the Azure Cli cli to have repeatable tasks. I also assume you use mac or Linux since the variables used were written for Bash.\nDon\u0026rsquo;t spin up a full-fledged GPU instance for setting up the machine, you can use a free version for the initial image and when done, copy or resize the instance.\nSetup #Some basic info for this tutorial, we use an initial Size of Standard_B2s and East US as the default location.\nLets first create a resource group, if you haven\u0026rsquo;t one already.\nFind a location and verify the size you want exist in the location\naz account list-locations az vm list-sizes -l ${azure_region} Create a resource group\naz group create --name ${resource_group} --location ${azure_region} Now create the VM, make sure you setup the storage-sku for later rezise compatibility.\naz vm create \\ --resource-group ${resource_group} \\ --name ${vm_name} \\ --image UbuntuLTS \\ --admin-username ${USERNAME} \\ --ssh-key-values \u0026lt;put the path or paths to your .pub ssh key here\u0026gt; \\ --size Standard_B2ms --storage-sku StandardSSD_LRS Connect to your VM #To find the public IP address of your VM, the response of the cli has it, also you can get it as follows:\naz vm show -d -g ${resource_group} -n ${vm_name} --query publicIps -o tsv Remember the machine the IP adress can change unless you set a pay static IP\nSSH to your instance using you .pub key\nssh \u0026lt;public_ip\u0026gt; Install drivers # Upgrade all, it is a good idea to restart after.\nsudo apt -qq update \u0026amp;\u0026amp; sudo apt -yqq upgrade sudo restart Setup the repositories and drivers, see that we dont need the actual GPU on the VM just yet. check the nvidia website for any updates on these steps:\nwget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-ubuntu1804.pin sudo mv cuda-ubuntu1804.pin /etc/apt/preferences.d/cuda-repository-pin-600 sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub sudo add-apt-repository \u0026quot;deb http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/ /\u0026quot; sudo apt-get -qq update sudo apt-get -yqq install cuda While you wait you could write your cuda code on a new terminal.\nSetup The Bash environment\necho export PATH=/usr/local/cuda-10.2/bin:/usr/local/cuda-10.2/NsightCompute-2019.1'${PATH:+:${PATH}}' \u0026gt;\u0026gt; ~/.bashrc echo export LD_LIBRARY_PATH=/usr/local/cuda-10.2/lib64'${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}'\u0026gt;\u0026gt;~/.bashrc Power off\nsudo poweroff Running on a GPU instance #Warning: Please don\u0026rsquo;t forget to deallocate your instance it will keep billing you if you don\u0026rsquo;t.\nThe new size we will be using is:\nsize = Standard_NC6_Promo Deallocate the VM for resizing\naz vm deallocate --name ${vm_name} --resource-group ${resource_group} Resize the VM\naz vm resize --resource-group ${resource_group} --name ${vm_name} --size ${size} Start the VM\naz vm start --resource-group ${resource_group} --name ${vm_name} Get the new IP\naz VM show -d -g ${resource_group} -n ${vm_name} --query publicIps -o tsv Testing it works # Check what NVIDIA card you have\nnvidia-smi Copy a cuda code, for example https://github.com/advt3/ParallelProgramming/blob/master/cuda/enumerate.cu\nwget https://raw.githubusercontent.com/advt3/ParallelProgramming/master/cuda/enumerate.cu Compile\nnvcc -o enumerate enumerate.cu Run\n./enumerate Shutdown and deallocate\naz vm deallocate --name ${vm_name} --resource-group ${resource_group} Verify the status of your VM\naz vm list -d -o table Cool we did it, I will recomend you scrip this steps so is a bit more natural an can be done often.\nFinal Thoughts #In this post, we have explained the complete environment setup using the latest Cuda driver and ubuntu 18.04, however, you can find a more complete but outdated description on the Microsoft documentation.\nIt will be nice to automatize the code running on a CI/CD pipeline so you can run your code only for the specific needed time. We will see when we have the time to test it.\nHappy coding!\nReferences # https://docs.microsoft.com/en-us/azure/virtual-machines/linux/n-series-driver-setup https://www.techpowerup.com/gpu-specs https://developer.nvidia.com/cuda-zone https://docs.microsoft.com/en-us/azure/virtual-machines/nc-series ","date":"May 28, 2020","permalink":"/posts/cloud_gpu/","section":"Posts","summary":"\u003ch1 id=\"introduction\" class=\"relative group\"\u003eIntroduction \u003cspan class=\"absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100\"\u003e\u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700\" style=\"text-decoration-line: none !important;\" href=\"#introduction\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\u003c/span\u003e\u003c/h1\u003e\u003cp\u003eWriting an algorithm employing Cuda does not require a state of the art GPU. However, to reap the benefits of massively\nparallel processing it is required a powerful one. How about using a cloud instance?.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eWriten by: Santiago Hurtado\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\n\n\n\n\n\n\n  \n  \n\u003cfigure\u003e\u003cimg src=\"/img/rack.jpg\" alt=\"rack\" class=\"mx-auto my-0 rounded-md\" /\u003e\n\u003c/figure\u003e\n\u003c/p\u003e\n\u003cp\u003eLet\u0026rsquo;s do some quick numbers first:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eAn \u003ca href=\"https://www.intel.com/content/dam/support/us/en/documents/processors/APP-for-Intel-Core-Processors.pdf\" target=\"_blank\" rel=\"noreferrer\"\u003eIntel i7-9700f\u003c/a\u003e\nprocessor should deliver 384 Gflops\nat a price of\nabout \u003ca href=\"https://www.amazon.com/Intel-i7-9700-Desktop-Processor-LGA1151/dp/B07S6CRLVD/ref=sr_1_4?dchild=1\u0026amp;keywords=intel\u0026#43;i7-9700f\u0026amp;qid=1590652540\u0026amp;sr=8-4\" target=\"_blank\" rel=\"noreferrer\"\u003e400 USD\u003c/a\u003e\u003c/p\u003e","title":"Using Cloud instances for your Cuda loads"},{"content":" Introduction #Having proper foundations in software usually translates on having a software project easy to extend and faster to develop. On its core, software engineering provides tools to improve: Business Value, Code Quality, and Process Predictability (Our primary focus).\nOur goal is to introduce Software Engineering practices, for the usage of Cloud and Edge Computing for video processing.\nWriten by: Santiago Hurtado\nEnvironment #We used a Linux environment tested on a bare-metal machine and also using docker containers. Docker provides good process predictability, and the benchmarks are run on the bare-metal machine for performance reasons.\nThe specifications used are the following:\nUbuntu 20.04 Docker Community Edition(CE)v19 Nvidia Cuda 10.2 Nvidia Container Toolkit OpenCV 4, CMake 3, C++, Boost 1.7, GTest and Python 3.8, Numpy *Nvidia Quadro P400 GPU *The P400 seems to be an excellent card to learn Cuda since it provides an architecture 6.1 and is not expensive. However, to have higher performance is recommended an Nvidia GeForce RTX 2060 GPU, at the least. https://timdettmers.com/2019/04/03/which-gpu-for-deep-learning/\nSystem Installation #Ubuntu 20.04 LTS #Installing ubuntu 20.04 LTS and Cuda together is very straight forward, go to https://ubuntu.com/download/desktop, there are many resources can be found. Many Linux distros have similar installation steps, for instance, CentOS and Redhat should be pretty much the same.\nInstall the Linux distro on your machine, update and upgrade your system and restart, Cuda # The Nvidia drivers should have been already installed, to verify run\nnvidia-settings When installing a new driver on the kernel be aware that new computers might have security for the UEFI boot https://wiki.ubuntu.com/UEFI/SecureBoot/DKMS.\nNext step is to install the Cuda SDK, for now, we will use the ubuntu 1804 support\nwget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-ubuntu1804.pin sudo mv cuda-ubuntu1804.pin /etc/apt/preferences.d/cuda-repository-pin-600 sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub sudo add-apt-repository \u0026quot;deb http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/ /\u0026quot; sudo apt-get update sudo apt-get install cuda Add to your .bashrc\nexport PATH=/usr/local/cuda-10.2/bin:/usr/local/cuda-10.2/NsightCompute-2019.1${PATH:+:${PATH}} export LD_LIBRARY_PATH=/usr/local/cuda-10.2/lib64 ${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}} The last step is to restart the system and check that everything works. Run the command and check that you see your card on the screen.\nnvidia-smi There are detailed instructions on the Cuda quick start https://docs.nvidia.com/cuda/cuda-quick-start-guide/index.html\nDockers #If you like, now you can install the docker-ce (community edition) on your system. For ubuntu 20.04 LTS there is already a package on the main package repository so the only step needed is to run the following command.\nsudo apt install docker.io Afterward, install the repository for the nVidia docker from https://github.com/NVIDIA/nvidia-docker. *Again we will use the ubuntu 18.04 repositories Instead of \u0026ldquo;distribution=$(. /etc/os-release;echo $ID$VERSION_ID)\u0026rdquo; set to ubuntu1804\nexport distribution=ubuntu18.04 curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list sudo apt-get update \u0026amp;\u0026amp; sudo apt-get install -y nvidia-container-toolkit sudo systemctl restart docker Development Libraries #Install OpenCV #To install OpenCV we will use the master from the git repository, you can select a specific version if you prefer. Here is the guide we used. Note: At this moment Cuda 10.02 can use only gcc 8, the ubuntu 20.04 default is 9.\n#[compiler] sudo apt-get install -y build-essential #[required] sudo apt-get install -y cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev #[optional] sudo apt-get install -y libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libdc1394-22-dev #[Cuda support] https://stackoverflow.com/questions/6622454/cuda-incompatible-with-my-gcc-version sudo apt-get install gcc-8 g++-8 sudo ln -s /usr/bin/gcc-8 gcc sudo ln -s /usr/bin/g++-8 g++ cd /usr/local/cuda-10.2/bin sudo rm gcc sudo rm g++ sudo ln -s /usr/bin/gcc-8 gcc sudo ln -s /usr/bin/g++-8 g++ cd /opt/libs git clone https://github.com/opencv/opencv.git git clone https://github.com/opencv/opencv_contrib.git mkdir opencv_build cd opencv_build cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D BUILD_PROTOBUF=OFF -D UPDATE_PROTO_FILES=ON -D BUILD_PNG=ON -D BUILD_JPEG=ON -D INSTALL_C_EXAMPLES=OFF -D WITH_CUDA=ON -DCUDA_HOST_COMPILER=/usr/bin/gcc-8 -D BUILD_EXAMPLES=OFF -D BUILD_PERF_TESTS=ON -D BUILD_opencv_java=OFF -D BUILD_opencv_python2=OFF -D BUILD_opencv_python3=ON -D INSTALL_PYTHON_EXAMPLES=OFF -D WITH_TBB=ON -D WITH_FFMPEG=ON -D OPENCV_EXTRA_MODULES_PATH=\u0026quot;../opencv_contrib/modules\u0026quot; ../opencv make -j 8 Install Boost 1.72 #To install boost you will not require many dependencies but if you want Boost.python you will need to have installed Python.\nsudo apt install python3-dev python3-numpy cd ~/Downloads wget https://dl.bintray.com/boostorg/release/1.72.0/source/boost_1_72_0.tar.gz tar xvfz boost_1_72_0.tar.gz -C /opt/libs cd /opt/libs/boost_1_72_0 This step is for Boost.Python, verify that you have all the paths correct and spaces.\n./bootstrap.sh --with-libraries=python,thread,log,system,chrono --with-python-version=3.8 echo \u0026quot;using python : 3.8 : /usr/bin/python3.8 : /usr/include/python3.8 : /usr/lib ;\u0026quot; \u0026gt;\u0026gt; project-config.jam ./b2 headers ./b2 sudo ./b2 install Install Gtest #There are many ways to install gTest, we use the package included in ubuntu.\nlibgtest-dev cd /tmp mkdir build mkdir /opt/libs/gtest cd build cmake /usr/src/googletest make \u0026amp;\u0026amp; sudo make install cd \u0026amp;\u0026amp; rm -rf /tmp/build Now you have a development system ready and there are plenty of examples out there. You can start with the nVidia Samples https://github.com/NVIDIA/cuda-samples, OpenCV https://docs.opencv.org/master/examples.html, and Boost https://theboostcpplibraries.com/.\nHappy coding!\nReferences #List of references presented on this post:\nhttps://timdettmers.com/2019/04/03/which-gpu-for-deep-learning/\nhttps://ubuntu.com/download/desktop\nhttps://wiki.ubuntu.com/UEFI/SecureBoot/DKMS\nhttps://developer.nvidia.com/cuda-toolkit\nhttps://docs.nvidia.com/cuda/cuda-quick-start-guide/index.html\nhttps://docs.docker.com/install/linux/docker-ce/ubuntu/\nhttps://github.com/NVIDIA/nvidia-docker\nhttps://www.packtpub.com/eu/application-development/building-computer-vision-projects-opencv-4-and-c\nhttps://www.packtpub.com/eu/application-development/cmake-cookbook\nhttps://github.com/NVIDIA/cuda-samples\n","date":"April 27, 2020","permalink":"/posts/development_environment/","section":"Posts","summary":"\u003cp\u003e\n\n\n\n\n\n\n  \n  \n\u003cfigure\u003e\u003cimg src=\"/img/construction.jpg\" alt=\"Building\" class=\"mx-auto my-0 rounded-md\" /\u003e\n\u003c/figure\u003e\n\u003c/p\u003e\n\u003ch2 id=\"introduction\" class=\"relative group\"\u003eIntroduction \u003cspan class=\"absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100\"\u003e\u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700\" style=\"text-decoration-line: none !important;\" href=\"#introduction\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\u003c/span\u003e\u003c/h2\u003e\u003cp\u003eHaving proper foundations in software usually translates on having a software project easy to extend and faster to\ndevelop. On its core, software engineering provides tools to improve: Business Value, Code Quality, and Process\nPredictability (Our primary focus).\u003c/p\u003e\n\u003cp\u003eOur goal is to introduce Software Engineering practices, for the usage of Cloud and Edge Computing for video processing.\u003c/p\u003e","title":"Development environment on Ubuntu 20.04 LTS"},{"content":"Following our Essential environment setup, We implement our first C++ and Cuda application modules. We describe on this post the steps needed to build and integrate the Cuda system with C++ and Python using CMake.\nWriten by: Santiago Hurtado\nThe explanations come with working code, so we describe only the key sections. We encourage you to get the code an play around. If any questions arise please don\u0026rsquo;t hesitate to contact us by dm on Twitter.\nBuilding C/C++ Applications with CMake is one of the most widely used approaches, we decide to build this prototype using it for better reproducibility. Our system consists of three implementations: Serial, Multicore and Parallel, it also includes python interfaces and unit tests. The design at the moment supports the compilation of most of the modules separately, however makes the compilation take longer.\nBuilding with CMake #Main CMake #On the main Cmake we use the include directory to have the common headers:\ninclude_directories(. include) We also define the dependency on CUDA and C++ 14, since it is what Cuda supports at the moment.\nset(CMAKE_CXX_STANDARD 14) set(CMAKE_CUDA_STANDARD 14) Don\u0026rsquo;t forget to enable testing to be able to use ctest\nenable_testing() Serial CMake #The next step is to have our serial implementation working. For this the key line needed for Boost Python to work is:\noption(BUILD_SHARED_LIBS \u0026quot;Build libraries as shared as opposed to static\u0026quot; ON) Multicore CMake #To have multithread support on the multicore implementation we use Boost Threads library.\nfind_package(Boost 1.70.0 COMPONENTS thread REQUIRED) And also we have the same line to build shared libs.\noption(BUILD_SHARED_LIBS \u0026quot;Build libraries as shared as opposed to static\u0026quot; ON) Parallel CMake #For Cuda support using CMake we set the project as Cuda and allow the separated compilation.\nfind_package(CUDA REQUIRED) project(parallel_math_lib CUDA) set_target_properties(${PROJECT_NAME} PROPERTIES CUDA_SEPARABLE_COMPILATION ON) Also we present automatically what architecture for cuda is supported by our device.\ncuda_select_nvcc_arch_flags(ARCH_FLAGS \u0026quot;Auto\u0026quot;) list(APPEND CUDA_NVCC_FLAGS ${ARCH_FLAGS}) Python CMake #Having the compilation of Boost Python with CMakecode working proved to be tricky. It requires to have libraries with the proper versions specified, found by the Boost Python build, and after by CMake. On the docker build Essential environment setup is presented how to build Boost with support for python 3.6.\nfind_package(Boost 1.70.0 COMPONENTS thread python REQUIRED) find_package(PythonInterp 3.6 REQUIRED) find_package(PythonLibs 3.6 REQUIRED) include_directories(${BOOST_INCLUDE_DIRS}) The last piece is to have the lib build without the lib prefix:\nset_target_properties(serial_math PROPERTIES PREFIX \u0026quot;\u0026quot;) The lib name has to match the module on the C++ Code\nBOOST_PYTHON_MODULE (serial_math){ }; The coolest part is that you can run python tests(or any executable), the key section is to tell python where to find the .o\nadd_test( NAME python_serial_test COMMAND ${CMAKE_COMMAND} -E env MATH_MODULE_PATH=$\u0026lt;TARGET_FILE_DIR:serial_math\u0026gt; ${PYTHON_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/tests/serial_test.py ) For python you need to append the path, a easy option is inside the python code\nsys.path.append(os.getenv('MATH_MODULE_PATH')) Unit Testing Build #Lastly to have the unit test running on CMake we need to have install gtest, there are recipes that can install the gtest as well, but we try to keep it simple.\nWe need support for C++(gcc) and Cuda(nvcc)\nproject(tests CXX CUDA) set(CMAKE_CXX_STANDARD 14) find_package(GTest REQUIRED) Include the path of the gtest libraries\ninclude_directories(${GTEST_INCLUDE_DIRS}) Add an executable by module\nadd_executable(serial_${PROJECT_NAME} serial_tests.cpp) Linking, dont forget that gtest needs pthread\ntarget_link_libraries(serial_${PROJECT_NAME} ${GTEST_LIBRARIES} pthread serial_math_lib) Add the tests to ctest to be executed on the \u0026lsquo;make test\u0026rsquo; command\nadd_test( NAME serial_${PROJECT_NAME} COMMAND $\u0026lt;TARGET_FILE:serial_${PROJECT_NAME}\u0026gt; ) And repeat for each unit test file\nDont forget to enable the tests\nenable_testing() Running the build #We build everything, cross our fingers and run the tests.\nCreate a build folder\nmkdir build Verify all the dependencies\ncd build cmake .. Build and test\nmake make test If everything works all six tests should pass. Troubleshooting # To debug the make phase use\nmake VERBOSE=1 To show the complete tests use\nctest -V Conclusion #When working on having performant software it is important to understand the different trade-offs. You must always perform benchmarks and stress tests, find where the biggest bottlenecks and check the different opportunities to improve.\nThe code done is far from optimal. If you are looking for an optimized solution is recommended to use libraries such as Cublas or Thrust, Where the theoretic performance of the device can be achieved.\nWe assume the main applications are written in Std C++ that is why the unit tests are developed in C++ using GTest. Considering that Python is one of the most widely used multipurpose programming languages, it makes sense to provide an interface to be used easily by the end-user.\nHappy building!\nReferences # https://devblogs.nvidia.com/building-cuda-applications-cmake/ ","date":"February 1, 2020","permalink":"/posts/running_cuda_cpp_python/","section":"Posts","summary":"\u003cp\u003eFollowing our \u003ca href=\"https://advt3.com/posts/development_environment/\" target=\"_blank\" rel=\"noreferrer\"\u003eEssential environment setup\u003c/a\u003e, We implement our first\nC++ and Cuda \u003ca href=\"https://github.com/advt3/ParallelProgramming/tree/master/parallel_math/\" target=\"_blank\" rel=\"noreferrer\"\u003eapplication modules\u003c/a\u003e. We describe\non this post the steps needed to build and integrate the Cuda system with C++ and Python using CMake.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eWriten by: Santiago Hurtado\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\n\n\n\n\n\n\n  \n  \n\u003cfigure\u003e\u003cimg src=\"/img/highway-2025863_640.jpg\" alt=\"Building\" class=\"mx-auto my-0 rounded-md\" /\u003e\n\u003c/figure\u003e\n\nThe explanations come with working code, so we describe only the key sections. We encourage you to get the code an play\naround. If any questions arise please don\u0026rsquo;t hesitate to contact us by dm on \u003ca href=\"https://twitter.com/wakesanti\" target=\"_blank\" rel=\"noreferrer\"\u003eTwitter\u003c/a\u003e.\u003c/p\u003e","title":"Building a Crossplatform Cuda application"},{"content":"Santiago Hurtado # Professional Profile #Principal Architect defining Infrastructure as a Product strategy and technical governance for cloud-native distributed infrastructure. I establish lean architectural blueprints bridging embedded Linux with Kubernetes fleet management, delivering secure, compliant systems aligned with IEC 62443, CRA, and NIS2.\nCareer Highlights # Global Edge Deployment at Scale: Architected an automated GitOps and OTA provisioning pipeline that reduced remote device onboarding from days to minutes. Enterprise Kubernetes Platform Design: Designed a Kubernetes Operator for enterprise MQTT messaging infrastructure, enabling seamless rolling upgrades. Stakeholder Leadership: Led cross-functional collaboration between vendors and engineering teams to translate customer needs into scalable technical solutions. Security by Design: Implemented Zero Trust principles and IEC 62443-aligned controls to harden distributed operations in regulated environments. ","date":null,"permalink":"/about/","section":"Engineering Infrastructure as a Product","summary":"\u003ch1 id=\"santiago-hurtado\" class=\"relative group\"\u003eSantiago Hurtado \u003cspan class=\"absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100\"\u003e\u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700\" style=\"text-decoration-line: none !important;\" href=\"#santiago-hurtado\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\u003c/span\u003e\u003c/h1\u003e\u003chr\u003e\n\u003ch1 id=\"professional-profile\" class=\"relative group\"\u003eProfessional Profile \u003cspan class=\"absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100\"\u003e\u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700\" style=\"text-decoration-line: none !important;\" href=\"#professional-profile\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\u003c/span\u003e\u003c/h1\u003e\u003cp\u003ePrincipal Architect defining Infrastructure as a Product strategy and technical governance for cloud-native distributed infrastructure. I establish lean architectural blueprints bridging embedded Linux with Kubernetes fleet management, delivering secure, compliant systems aligned with \u003cstrong\u003eIEC 62443, CRA, and NIS2\u003c/strong\u003e.\u003c/p\u003e\n\u003chr\u003e\n\u003ch1 id=\"career-highlights\" class=\"relative group\"\u003eCareer Highlights \u003cspan class=\"absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100\"\u003e\u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700\" style=\"text-decoration-line: none !important;\" href=\"#career-highlights\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\u003c/span\u003e\u003c/h1\u003e\u003cul\u003e\n\u003cli\u003e\u003cstrong\u003eGlobal Edge Deployment at Scale:\u003c/strong\u003e Architected an automated GitOps and OTA provisioning pipeline that reduced remote device onboarding from days to minutes.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eEnterprise Kubernetes Platform Design:\u003c/strong\u003e Designed a Kubernetes Operator for enterprise MQTT messaging infrastructure, enabling seamless rolling upgrades.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eStakeholder Leadership:\u003c/strong\u003e Led cross-functional collaboration between vendors and engineering teams to translate customer needs into scalable technical solutions.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eSecurity by Design:\u003c/strong\u003e Implemented Zero Trust principles and IEC 62443-aligned controls to harden distributed operations in regulated environments.\u003c/li\u003e\n\u003c/ul\u003e","title":"About Me"},{"content":"Key Projects #Infrastructure-as-a-Product (Orise) #Principal Systems Architect\nArchitecting a modular infrastructure framework and automated delivery pipeline for global edge deployments.\nGlobal Edge Deployment: Designed a GitOps and OTA provisioning pipeline that reduced remote device onboarding from days to minutes. Fleet Orchestration: Engineered a resilient delivery pipeline using FluxCD and Mender to automate reliable OS and application updates across 500+ sites. Security \u0026amp; Compliance: Aligned infrastructure delivery with NIS2 and IEC 62443 security standards. ","date":null,"permalink":"/projects/","section":"Engineering Infrastructure as a Product","summary":"\u003ch1 id=\"key-projects\" class=\"relative group\"\u003eKey Projects \u003cspan class=\"absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100\"\u003e\u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700\" style=\"text-decoration-line: none !important;\" href=\"#key-projects\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\u003c/span\u003e\u003c/h1\u003e\u003ch2 id=\"infrastructure-as-a-product-orise\" class=\"relative group\"\u003eInfrastructure-as-a-Product (Orise) \u003cspan class=\"absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100\"\u003e\u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700\" style=\"text-decoration-line: none !important;\" href=\"#infrastructure-as-a-product-orise\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\u003c/span\u003e\u003c/h2\u003e\u003cp\u003e\u003cem\u003ePrincipal Systems Architect\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eArchitecting a modular infrastructure framework and automated delivery pipeline for global edge deployments.\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\u003cstrong\u003eGlobal Edge Deployment:\u003c/strong\u003e Designed a GitOps and OTA provisioning pipeline that reduced remote device onboarding from days to minutes.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eFleet Orchestration:\u003c/strong\u003e Engineered a resilient delivery pipeline using FluxCD and Mender to automate reliable OS and application updates across 500+ sites.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eSecurity \u0026amp; Compliance:\u003c/strong\u003e Aligned infrastructure delivery with NIS2 and IEC 62443 security standards.\u003c/li\u003e\n\u003c/ul\u003e","title":"Key Projects"}]