BIPro

Analytical AI Agents with Looker’s Trusted Metrics, AI Powered Data Clean Rooms @media only screen and (max-width: 100%;} #pad-desktop {display: none !important;} } 👋 Hey ,Happy Thursday! 🍃Welcome to BIPro #72—Your Weekly Dose of Data Brilliance!This week’s newsletter is packed with insights to boost your data game:🔮 Data Insights & Tools◉ Pipeline Perfection: Build Efficient Data Pipelines with Prefect◉ Taming Outliers: Handle Dataset Outliers Using Pandas◉ Regex Wizardry: 5 Tips for Data Cleaning with Regular Expressions◉ NumPy Secrets: Solve Nonlinear Equations with NumPy◉ Docker Essentials: Use Docker Volumes for Persistent Storage◉ Python Library Creation: A Beginner’s Guide to Pip Install YOU⚡ Industry Highlights◉ Microsoft Fabric: August 2024 Update, Advanced Anomaly Detection, Custom Sparklens JARs, and CI/CD Capabilities◉ AWS BI: Batch Data Processing with AWS Lambda, Zero Copy Data Sharing, and Unified Analytics◉ Google Cloud: Trusted Metrics with Looker, AI-Powered Data Clean Rooms◉ Tableau: New Features in Tableau 2023.1, Creative Collaboration with Deloitte, Salesforce CRM Integration, and Tableau Portals✨ Fresh Reads◉ Microsoft Power BI Cookbook by Greg Deckler & Brett Powell: Master data transformation◉ Big Data on Kubernetes by Neylson Crepalde: Build scalable data solutions◉ Big Data Using Hadoop and Hive by Nitin Kumar: Advanced Hadoop & Hive techniques◉ Tableau Masterclass 2024 by Nikolai Schuler: From Basics to Advanced Analytics💡 BI Community Scoop◉ Microsoft Fabric Security: Object, Column, and Row Level◉ SQL Performance: Local Variables and Metadata Insights◉ Data Replication & Integration: Change Data Capture Strategies◉ SQL Server Logs: How to Access Error LogsStay sharp and keep innovating with BIPro!Calling All Data & BI Enthusiasts!Do you dream of sharing your insights and building your reputation in the Data & BI community? Contribute to our new column in the Packt BIPro newsletter! Share your experiences, discuss new BI tools, or ask questions. Gain recognition among 37,000 BI professionals. Reply with your Google Docs article or use our weekly feedback form. Enjoy a free PDF of "Interactive Data Visualization with Python - Second Edition" for participating. Click reply or share your content today!Share your thoughts and opinions here!Cheers,Merlyn ShelleyEditor-in-Chief, PacktSign Up|Advertise|Archives✨Fresh Off the Press: Exclusive New Titles Just for You!We’re thrilled to bring you this week’s must-have new releases, straight from the experts to your bookshelf! Whether you're eager to enhance your skills or explore new horizons, now is the perfect moment to add these invaluable resources to your collection.For a limited time, enjoy 30% off all eBooks at Packtpub.com. These books are thoughtfully crafted by industry insiders with hands-on experience, offering unique insights you won’t find anywhere else.Don’t let these Packt-exclusive deals slip away—seize the opportunity to learn from the best at an unbeatable price!Buy now at $29.99 $43.99Microsoft Power BI Cookbook: Convert raw data into business insights with updated techniques, use cases, and best practices, Third EditionBy Greg Deckler, Brett PowellIf you're looking to elevate your data game, the latest edition of the Power BI Cookbook is your perfect guide. Whether you're a seasoned BI developer or just diving into data analytics, this updated resource offers:◾ Deeper insights through Microsoft Data Fabric for robust data strategies.◾ Simplified creation of Hybrid tables, scorecards, and shared cloud connections.◾ Enhanced visualization tools to turn complex data into clear, actionable insights.With step-by-step guidance, this book equips you to navigate the evolving landscape of Power BI and stay ahead with the latest innovations. Perfect for refining skills or mastering new ones. Grab the Power BI Cookbook 3rd Edition now for just $29.99!Buy now at $21.99$31.99Big Data on Kubernetes: A practical guide to building efficient and scalable data solutionsBy Neylson CrepaldeIf you're navigating the complexities of big data in a cloud environment, Big Data on Kubernetes is your guide to mastering scalable, resilient data pipelines. This book offers practical insights to:◾ Seamlessly integrate Kubernetes with popular tools◾ Optimize big data pipelines for peak performance◾ Build end-to-end solutions with Spark, Airflow, and KafkaWhether you're just starting out or looking to enhance your expertise, this resource will empower you to handle real-world data challenges confidently. Grab it now at just $21.99 – save $10 on your essential big data guide!Buy now at $37.99$54.99Big Data Using Hadoop and Hive: Master Big Data Solutions with Hadoop and HiveBy Mercury Learning and Information, Nitin KumarLooking to master big data? This new release is your go-to guide for diving deep into Hadoop 3 and Hive 3.x. You’ll get:◾ Comprehensive coverage of Hadoop 3 and Hive 3.x◾ Real-world examples and sample code for practical applications◾ Advanced insights into YARN, MapReduce, and data compressionPerfect for developers and engineers, this book takes you from the basics of big data to advanced data management techniques, ensuring you can confidently set up, configure, and optimize Hadoop and Hive to tackle big data challenges. Ready to level up your data game? This guide is your essential companion. Unlock the power of big data for just $37.99—save $17 on this must-have first edition!Buy nowTableau Masterclass - 2024: Master Tableau: From Basics to Advanced Analytics [Video]By Nikolai SchulerReady to transform your data into compelling stories? This new release is your guide to mastering Tableau—from basic connections to advanced visualizations. Learn to blend data, craft dynamic dashboards, and publish your insights with confidence. Whether you're starting out or leveling up, this course equips you with everything you need to excel in data visualization.Key Benefits:◾ Master Tableau from basics to advanced features◾ Learn effective data visualization and storytelling techniques◾ Create, design, and publish interactive dashboardsGet ready to tackle any data visualization challenge with this detailed video guide! Become a Tableau Pro: 8-Hour Course for $109.99—Watch Now!🚀Business Intelligence Toolkit➤ PrefectHQ/prefect: Prefect simplifies Python data pipeline orchestration, transforming scripts into dynamic workflows that react to changes and ensure resilience.➤ airbytehq/airbyte: Airbyte, an open-source data integration platform, offers 300+ connectors for seamless ELT pipelines between diverse data sources and destinations.➤ argoproj/argo-workflows: Argo Workflows orchestrates parallel jobs on Kubernetes via container-native workflows, supporting DAGs and accelerating compute-intensive tasks like ML and data processing.➤ dagster-io/dagster:Dagster is a cloud-native data pipeline orchestrator with integrated lineage, observability, declarative programming, and robust testability across the lifecycle.➤ Avaiga/taipy: Taipy simplifies web app development for data scientists & ML engineers using Python, focusing on AI algorithms with no extra languages.Access 100+ data tools in this specially curated blog, covering everything from data analytics to business intelligence—all in one place. Check out "Top 100+ Essential Data Science Tools & Repos: Streamline Your Workflow Today!" on PacktPub.com. Forward to a Friend!🔮Data Visualization with Python➤Building Data Pipeline with Prefect: The tutorial introduces Prefect, a modern workflow orchestration tool, by building a data pipeline with Pandas and comparing it to a Prefect workflow. It covers task orchestration, deployment, and monitoring of workflows, demonstrating Prefect's features for efficient workflow management and observability in MLOps.➤ How to Handle Outliers in Dataset with Pandas? This blog discusses the detection and handling of outliers in datasets, explaining their impact on data analysis and models, and explores various techniques such as removal, capping, imputation, and transformation to manage outliers effectively.➤ 5 Tips for Using Regular Expressions in Data Cleaning: This blog explains how to use regular expressions in Python for text processing and data cleaning, covering tasks like removing unwanted characters, extracting specific patterns, replacing text, validating data formats, and splitting strings, including examples with Pandas.➤ How to Use NumPy to Solve Systems of Nonlinear Equations? This blog explains nonlinear equations, their importance in modeling real-world problems, and how to solve them using Python's NumPy and SciPy. It covers defining equations, making initial guesses, solving systems, and visualizing results in 2D and 3D.➤ How to Use Docker Volumes for Persistent Data Storage? This blog explains how to use Docker volumes to persist data in PostgreSQL containers. It covers creating a Docker volume, running a PostgreSQL container with the volume, verifying data persistence, and ensuring data remains intact after stopping and restarting the container.➤ Pip Install YOU: A Beginner’s Guide to Creating Your Python Library. This blog provides a step-by-step guide for creating, structuring, and distributing custom Python libraries. It covers everything from project initialization, module creation, and adding tests to publishing the library on PyPI for others to use.⚡Stay Informed with Industry HighlightsMicrosoft Fabric➤ Microsoft Fabric August 2024 Update: The August 2024 Fabric update introduces key features: managing V-Order in Fabric Warehouses, ML experiment monitoring from the Monitor Hub, and streamlined Azure connectivity in Data Pipeline. It highlights the European Fabric Community Conference, new Copilot features, visual-level format strings in Power BI, and the Fabric Influencers Spotlight. Additionally, it stresses the importance of browser upgrades for Power BI and offers new certification and community engagement opportunities.➤ Advanced Time Series Anomaly Detector in Fabric: Azure's Anomaly Detector, retiring in October 2026, enabled time series anomaly detection using advanced algorithms. This blog outlines a migration strategy to Microsoft Fabric, leveraging similar algorithms with added benefits like easier model management, seamless data integration, and expanded detection capabilities using Fabric's native tools and the new time-series-anomaly-detector package.➤ Building a Custom Sparklens JAR for Microsoft Fabric: This blog explains how to build a Sparklens JAR compatible with Spark 3.X for profiling Microsoft Fabric Spark Notebooks. It covers modifying build and configuration files, updating code for Spark 3.X compatibility, and compiling and packaging the JAR for use in Microsoft Fabric.➤ Exploring CI/CD Capabilities in Microsoft Fabric: A Focus on Data Pipelines. This blog explores Microsoft Fabric's CI/CD features, focusing on automating and managing data integration and analytics processes. It highlights Git integration, deployment pipelines, and workspace setup for streamlined continuous integration and deployment. The blog also provides a step-by-step guide for setting up and operating CI/CD processes in Microsoft Fabric using Azure DevOps and Git.AWS BI➤ Efficiently processing batched data using parallelization in AWS Lambda: This post explains how to optimize AWS Lambda functions for efficient message processing by using techniques like batching and parallelization, enhancing resource utilization, reducing invocation times, and improving overall performance in high-volume data processing scenarios.➤ Harness Zero Copy data sharing from Salesforce Data Cloud to Amazon Redshift for Unified Analytics: This article discusses how Salesforce and Amazon have collaborated to enable seamless, bidirectional Zero Copy data sharing between Salesforce Data Cloud and Amazon Redshift. It details how this integration allows analytics teams to access and analyze unified customer data without the need for traditional ETL processes, enhancing efficiency and accelerating insights.Google Cloud Data➤ Grounding Analytical AI Agents with Looker’s Trusted Metrics: This article explores how organizations can integrate AI, particularly Large Language Models (LLMs) like Gemini, with Google Cloud's data tools like Looker to enhance Business Intelligence (BI). By combining AI with Looker's semantic layer, companies can offer users intuitive, AI-driven insights, simplifying data access and decision-making processes. The article highlights the ease and effectiveness of training AI models to align with specific business needs, lowering barriers to data-driven decision-making and enabling faster, more accurate analytics.➤ Modern Marketer’s Strategic Advantage AI Powered Data Clean Rooms: This article explains how businesses can use Google BigQuery data clean rooms to securely analyze and share sensitive customer data across organizations, driving insights and collaboration. It highlights the importance of AI-powered data clean rooms for modern marketers to unlock insights, fuel innovation, and enhance customer experiences while maintaining data privacy and security.Tableau➤ What's New in Tableau 2023.1? The Tableau 2023.1 feature update includes significant enhancements such as improved Tableau-Slack integration, dynamic axis titles, Accelerator Data Mapping, and advanced management features like Identity Pools and RMT improvements. It also introduces new tools for developers, web authoring improvements, expanded data connectivity options, and enhanced data preparation and management capabilities.➤ Building a Culture of Creative Collaboration with Deloitte and Tableau: This article discusses the growing data and analytical skills gap in the AI-driven business landscape. It highlights the collaboration between Salesforce and Deloitte to bridge this gap through innovative talent development programs like Deloitte Viz Games, which foster a data-driven culture and enhance data literacy, analytics, and creative collaboration among employees.➤ Salesforce Embeds Tableau Pulse into its CRM: Salesforce has introduced Pulse for Salesforce, a version of Tableau Pulse integrated into Salesforce CRM, starting with Sales Cloud. Built on the Einstein 1 Platform, it leverages generative AI to provide personalized, contextual insights and metrics within users' workflows, enhancing data-driven decision-making and supporting daily business activities securely.➤ What Is a Tableau Portal? Tools, Benefits & Case Study. This article discusses Tableau Portals, customized web interfaces that integrate Tableau’s data visualization into a centralized, branded platform. It highlights the benefits of self-service analytics, centralized access, enhanced security, and improved customer experience. The article also details features like content management, alerts, and a case study showcasing the effectiveness of implementing Tableau Portals for client reporting. 💡What's the Latest Scoop from the BI Community?➤ Microsoft Fabric Warehouse Security: Object, Column and Row Level. The article addresses the challenge of implementing granular access control in Microsoft Fabric's data warehouse. It explores various security mechanisms like object-level, column-level, and row-level security to restrict sensitive data access. Additionally, it highlights limitations when users access data through Spark or OneLake, bypassing these controls.➤ SQL Local Variables and Performance Issues: The article discusses the potential negative impact of using local variables in T-SQL queries on performance. It explores how local variables can lead to inefficient query plans and offers solutions and workarounds to mitigate these issues, ultimately improving query performance.➤ SQL Metadata in sys.databases, sys.objects, sys.tables and sys.columns: The article explains how SQL Server metadata, which includes data about databases, tables, columns, and keys, can be accessed using sys schema catalog views like sys.databases, sys.objects, and sys.columns. It provides T-SQL examples to query and understand metadata, helping users efficiently manage and utilize SQL Server metadata for various database objects.➤ Data Replication and Change Data Capture for Data Integration: The article discusses the challenge of replicating real-time data from production databases to data products without impacting database performance. It introduces Integrate.io as a low-code platform offering high-velocity data replication using Change Data Capture (CDC) and ETL. The platform supports seamless data pipeline automation, scalability, and security, ensuring efficient real-time data integration for business intelligence applications.➤ Microsoft Fabric OneLake Role Based Access Control (RBAC): The article discusses how to implement granular access control in a Microsoft Fabric lakehouse using Role-Based Access Control (RBAC) in OneLake. This feature allows administrators to restrict access to specific folders or tables within a lakehouse, ensuring that users only access the data they are permitted to see.➤ How to Access the SQL Server Error Log? The article explains how to view SQL Server and SQL Agent error logs, highlighting three primary methods: using SQL Server Management Studio (SSMS) Log File Viewer, accessing logs via the system stored procedure `sp_readerrorlog`, and discussing when to use each method based on the need for speed, filtering, and custom log analysis.See you next time! Copyright (C) 2024 Packt Publishing. All rights reserved.Our mailing address is:Packt Publishing Grosvenor House 11 St Paul's Square Birmingham, West Midlands B3 1RB United KingdomWant to change how you receive these emails?You canupdate your preferencesorunsubscribe*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{line-height:0;font-size:75%} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}} @media only screen and (max-width: 100%;} #pad-desktop {display: none !important;} }

For a limited time, save on the best-selling books that will elevate your skills and knowledge! @media only screen and (max-width: 100%;} #pad-desktop {display: none !important;} }👋 Hello ,✨ Welcome to Packt’s Signature Series: New Titles Just Arrived!📚 We’re excited to present a new collection in our Signature Series, featuring the best-selling titles in the data industry. Packed with insights on Generative AI and multimodal systems, this collection is available for a limited time at 30% off both print and e-book formats. This offer ends Sunday, September 22nd. Don’t miss your chance to upskill and elevate your career. Let’s dive in!➽ Building LLM Powered Applications: This new titleis all about helping engineers and data pros use large language models (LLMs) effectively. It tackles key challenges like embedding LLMs into real-world apps and mastering prompt engineering techniques. You’ll learn to orchestrate LLMs with LangChain and explore various models, making it easier to create intelligent systems that can handle both structured and unstructured data. It’s a great way to boost your skills, whether you’re new to AI or already experienced! Start your free trial for access, renewing at $19.99/month.eBook $27.98 $39.99Print + eBook $34.98 $49.99➽ Python for Algorithmic Trading Cookbook: This bookis your go-to guide for using Python in trading. It helps you tackle key issues like acquiring and visualizing market data, designing and backtesting trading strategies, and deploying them live with APIs. You’ll learn practical techniques to gather data, analyze it, and optimize your strategies using tools like OpenBB and VectorBT. Whether you’re just starting or looking to refine your skills, this book equips you with the know-how to trade smarter with Python! Start your free trial for access, renewing at $19.99/month.eBook $27.98 $39.99Print + eBook $36.99 $49.99➽ Microsoft Power BI Cookbook - Third Edition: The Power BI Cookbook is your essential guide to mastering data analysis and visualization with Power BI. It covers using Microsoft Data Fabric, managing Hybrid tables, and creating effective scorecards. Learn to transform complex data into clear visuals, implement robust models, and enhance reports with real-time data. This updated edition prepares you for future AI innovations, making it a must-have for beginners and seasoned users alike! Start your free trial for access, renewing at $19.99/month.eBook $29.99 $43.99Print + eBook $41.98 $59.99➽ The Definitive Guide to Power Query (M): The Definitive Guide to Power Query (M) focuses on mastering data transformation with Power Query. It covers fundamental and advanced concepts through hands-on examples that address real-world problems. You'll learn the Power Query M language, optimize performance, handle errors, and implement efficient data processes. By the end, you'll have the skills to enhance your data analysis effectively! Start your free trial for access, renewing at $19.99/month.eBook $43.99Print + eBook $37.99 $54.99➽ Mastering PyTorch - Second Edition: This is your essential resource for building advanced neural network models with PyTorch. You'll explore tools like Hugging Face, fastai, and Docker, learning to create models for text, images, and music. With hands-on examples, you'll master training optimization, mobile deployment, and various network types, equipping you to tackle complex AI tasks using the PyTorch ecosystem! Start your free trial for access, renewing at $19.99/month.eBook $28.99 $41.99Print + eBook $40.99 $51.99➽ Unlocking the Secrets of Prompt Engineering: It'syour guide to mastering AI-driven writing with large language models (LLMs). It covers essential techniques and applications, from content creation to chatbots. With practical examples, you'll learn to generate product descriptions and tackle advanced uses like podcast creation. The book emphasizes ethical practices and optimization strategies, preparing you to leverage AI for improved writing, creativity, and productivity! Start your free trial for access, renewing at $19.99/month.eBook $24.99 $35.99Print + eBook $30.99 $44.99➽ ChatGPT for Cybersecurity Cookbook: Your essential guide to using AI in cybersecurity. It helps you automate tasks like penetration testing, risk assessment, and threat detection with ChatGPT. Each recipe provides step-by-step instructions for generating commands, writing code, and creating tools with the OpenAI API and Python. You'll explore innovative strategies and optimize workflows, gaining confidence in AI-driven techniques to excel in the rapidly evolving cybersecurity landscape! Start your free trial for access, renewing at $19.99/month.eBook $27.98 $39.99Print + eBook $34.98 $49.99➽ Mastering NLP from Foundations to LLMs:Your complete guide to Natural Language Processing (NLP) with Python. It covers the mathematical foundations of machine learning and essential topics like linear algebra and statistics. You'll learn to preprocess text, classify it, and implement advanced techniques, including large language models (LLMs). With practical Python code samples and insights into future trends, you'll gain the skills to tackle real-world NLP challenges confidently and effectively design ML-NLP systems! Start your free trial for access, renewing at $19.99/month.eBook $29.99 $42.99Print + eBook $46.99 $52.99➽ Learn Microsoft Fabric: This title is your essential guide to using Microsoft Fabric for data integration and analytics. It explores key features with real-world examples, helping you build solutions for lakehouses, data warehouses, and real-time analytics. You'll learn to effectively monitor your Fabric platform and cover workloads like Data Factory and Power BI. By the end, you'll be equipped to unlock AI-driven insights and navigate the analytics landscape confidently! Start your free trial for access, renewing at $19.99/month.eBook $24.99 $35.99Print + eBook $35.98 $44.99➽ Building Data-Driven Applications with LlamaIndex: This book is your comprehensive guide to leveraging Generative AI and large language models (LLMs). It addresses challenges like memory constraints and data gaps while teaching you to build interactive applications with LlamaIndex. You'll learn to ingest and index data, create optimized indexes, and query your knowledge base through hands-on projects. By the end, you'll be equipped to troubleshoot LLM issues and confidently deploy your AI-driven applications! Start your free trial for access, renewing at $19.99/month.eBook $24.99 $35.99Print + eBook $30.99 $44.99➽ OpenAI API Cookbook: This new title is all about using the OpenAI API to create smart applications. It helps engineers and data pros understand the basics, set up their API, and build tailored tools like chatbots and virtual assistants. You’ll learn practical recipes to enhance user experience and integrate AI into your workflows, making your projects more efficient and innovative! Start your free trial for access, renewing at $19.99/month.eBook $21.99 $31.99Print + eBook $27.98 $39.99Loved Those Titles? Check These Out!➽ Data Governance Handbook➽ Generative AI for Cloud Solutions➽ Data-Centric Machine Learning with Python➽ Modern Python Cookbook - Third EditionWe’ve got more great things coming your way—see you soon!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{line-height:0;font-size:75%} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}} @media only screen and (max-width: 100%;} #pad-desktop {display: none !important;} }

Build Polymorphic Associations in SQL Server with Foreign Keys, Data Control LanguageStop worrying about your to-do list.Zapier connects the apps you use every day, so you can focus on what matters most.Start working more efficiently -Create your free account today.Get started for freeSponsored🗞️Welcome to BIPro #87 – Your Weekly Business Intelligence Boost! 🚀Get ready for this week’s latest BI trends, strategies, and insights to fuel your data-driven success!📊 Data Trends That Matter◘ LlamaIndex Meets Cloud Power: Unlock better insights with LlamaIndex’s integration with AlloyDB and PostgreSQL Cloud SQL.◘ Revamping Supply Chains: How Rehrig Pacific leverages Amazon QuickSight for transformative analytics.◘ No-Code Wizardry: Open Interpreter makes BI accessible to everyone—no code required!◘ Direct Data Magic: A fresh approach to visualizing data straight from Numpy arrays.◘ Microsoft Fabric Gets the Green Light: Now FedRAMP High authorized—secure your BI in Azure Commercial.◘ The NOLOCK Paradox: Why “dirty reads” might just clean up your database performance.◘ JSON in PostgreSQL: Powerful, versatile, and essential for modern BI.Mastering Software Deployments at the Edge: A User’s Guide to Diverting DisasterSoftware delivery to dedicated edge devices is one of the most complex challenges faced by IT professionals today. While edge deployments come with inherent complications, it’s possible to avoid the pitfalls. With this guide in hand, a little planning, and the right tools and strategies in place, you can be confident you’ll never push a faulty update at scale.Read the GuideSponsored🔄 Transformations That Inspire◘ SAP HANA’s Safety Net: Google Cloud’s Backup and DR Service for enterprise peace of mind.◘ New AWS Datasets: 39 fresh additions to supercharge your analysis on the Registry of Open Data.◘ Data Security Simplified: A closer look at Data Control Language (DCL).◘ Real-Time BI Monitoring: Fabric Spark applications with live insights.◘ Microsoft SQL Server 2025: AI-ready database redefined—cloud to ground.◘ Smart Associations: Building polymorphic relationships in SQL Server.◘ Effortless Pipeline Management: Streamline Azure Data Factory pipelines in Microsoft Fabric.◘ PostgreSQL Optimization: Query smarter, not harder.⚡ Quick BI Wins◘ Firestore Migration Success: How HighLevel transitioned workloads with ease.◘ Save Big on AWS: Practical tips for effective cost optimization.◘ Real-Time Dashboards in a Snap: Seamlessly recreate your ADX dashboards in Fabric.◘ Structured Data Basics: Build a solid BI foundation with key principles.◘ Meet ChatGPT Pro: The next level in conversational AI.◘ Sora Is Here: Discover the new standard in AI tools.◘ DIY AI Training: Use Google Colab to train your own language models.🎤 Insights from BI Pros◘ AI Meets Strategy: Integrating AI and data science into your business roadmap.◘ Closing the Data Literacy Gap: A deep dive into the evolution and future of data skills.◘ GPS and Analytics: Bridging maps, kinematics, and BI for next-gen solutions.◘ Power BI Teams Update: What the ‘Teams activity analytics’ deprecation means for you.◘ From Code to Paper: Using GPT and Python to create scientific documents.◘ SQL vs. Spreadsheets: Building robust champion/challenger tests from scratch.Dive in and let this week’s insights supercharge your BI journey! 🚀Calling All Data & BI Enthusiasts!Do you dream of sharing your insights and building your reputation in the Data & BI community? Contribute to our new column in the Packt BIPro newsletter! Share your experiences, discuss new BI tools, or ask questions. Gain recognition among 37,000 BI professionals. Reply with your Google Docs article or use our weekly feedback form. Enjoy a free PDF of "Interactive Data Visualization with Python - Second Edition" for participating. Click reply or share your content today!Share your thoughts and opinions here! @media only screen and (max-width: 100%;} #pad-desktop {display: none !important;} }This is our final edition of BIPro for 2024, but don’t worry—we’ll be back with more insights and updates in January 2025. In the meantime, we’ve got a little holiday treat for you! Packt has some exciting offers lined up to help you boost your tech skills and get ready for an amazing new year! It’s the perfect opportunity to relax, learn something new, and stay ahead in your field. Keep an eye out for these special holiday deals!From all of us at the Packt Newsletters team, we wish you a joyful holiday season and a fantastic start to 2025. See you next year! 🎄✨Cheers,Merlyn ShelleyEditor-in-Chief, PacktPackt’s Signature Series: New Titles Just Arrived!📚➽Learn Microsoft Fabric: Explore Microsoft Fabric's features through real-world examples to build robust data analytics solutions, including lakehouses and data warehouses. Learn to monitor and manage your analytics system for flexibility, performance, and security, while leveraging AI-driven insights with Copilot integration. Start your free trial for access, renewing at $19.99/month.eBook $24.99 $35.99Print + eBook $44.99➽Microsoft Power BI Cookbook - Third Edition: Dive into Microsoft Data Fabric to enhance data strategies and gain deeper insights. Effortlessly create Hybrid tables and comprehensive scorecards while utilizing new visualization tools that transform complex data into clear, actionable charts and reports for effective decision-making in Power BI. Start your free trial for access, renewing at $19.99/month.eBook $29.99 $43.99Print + eBook $59.99➽Fundamentals of Analytics Engineering: Explore how analytics engineering aligns with your organization's data strategy while gaining insights from seven industry experts. Address common challenges faced by businesses and learn to implement scalable analytics solutions, from data ingestion to visualization, using industry-leading tools. Start your free trial for access, renewing at $19.99/month.eBook $24.99 $35.99Print + eBook $44.99➽Getting Started with DuckDB: Utilize DuckDB to efficiently load, transform, and query diverse data sources and formats. Gain hands-on experience with SQL, Python, and R for data analysis, while exploring how open-source tools and cloud services enhance DuckDB’s versatile capabilities in the data ecosystem. Start your free trial for access, renewing at $19.99/month.eBook $29.99 $43.99Print + eBook $54.99📊 Data Viz Trends Shaping the Future of Insights⫸ LlamaIndex integrates with AlloyDB and Cloud SQL for PostgreSQL: This blog dives into how AI agents, powered by LlamaIndex and Google Cloud integrations, are transforming application development. It highlights agentic RAG workflows, complex data parsing, and advanced knowledge retrieval, showcasing new possibilities for automating tasks like report generation and beyond.⫸ Rehrig Pacific Company transforms supply chain analytics with Amazon QuickSight: This blog highlights how Rehrig Pacific transformed its analytics with Amazon QuickSight. It explores how they overcame data growth challenges, rapidly deployed dashboards, embedded AI-driven analytics, and boosted customer satisfaction while planning future AI enhancements to scale operations efficiently.⫸ No Code, No Problem: How to Use Open Interpreter: This blog introduces Open Interpreter, a no-code tool that lets you control your computer with natural language commands. Learn how to install it, configure API keys, and use it for tasks like math calculations, defining functions, and data analysis effortlessly!⫸ Visualizing Data Directly from Numpy Arrays: This tutorial covers visualizing data in Python using NumPy and Matplotlib. Learn practical examples such as line plots for stock prices, scatter plots for height versus weight analysis, and 2D array heatmaps for temperature data to build essential data visualization skills.⫸ Microsoft Fabric approved as a Service within the FedRAMP High Authorization for Azure Commercial: Microsoft Fabric has achieved FedRAMP High Authorization for Azure Commercial, meeting rigorous security standards for US government agencies. This milestone enables federal organizations to securely adopt AI-powered tools to manage, connect, and analyze data while ensuring compliance.⫸ The Paradox of NOLOCK: How Dirty Reads Can Sometimes Be Cleaner. This blog explores the nuances of using NOLOCK in SQL Server. While often discouraged, NOLOCK can improve query speed by reading uncommitted data, which is useful for non-critical reports. It highlights strategic use cases, trade-offs, and when accuracy must take priority.⫸ JSON in PostgreSQL: This article provides a practical guide to using JSON in PostgreSQL, covering JSON data types, key operators, and functions. Learn to store, query, and manipulate JSON efficiently with examples of table creation, valid data insertion, and querying JSON fields.🔄 Real-World Transformation: How Gen BI Made Data Work⫸ Google Cloud Backup and DR Service for SAP HANA: This article explores Google Cloud's Backup and DR solution for SAP HANA, highlighting cost-effective cold disaster recovery strategies with Persistent Disk snapshots. Learn how integration with HANA Savepoints enables faster recovery, reduced storage costs, and simplified DR management.⫸ 39 new or updated datasets available on the Registry of Open Data on AWS: This article highlights the AWS Open Data Sponsorship Program, which democratizes access to over 100 petabytes of cloud-optimized datasets for public analysis. It features 39 newly released datasets, including medical imaging, climate, and geospatial data, fostering innovation and collaboration.⫸ Data Control Language (aka Security): This article explores the three SQL sub-languages: DDL, DML, and DCL emphasizing their interconnected roles in schema design, data manipulation, and privilege management. It highlights best practices, potential pitfalls, and the significance of thoughtful privilege allocation to ensure secure and effective database management.⫸ Monitor Fabric Spark applications using Fabric Real-Time Intelligence: This article explains how to set up a centralized Spark monitoring solution in Fabric using Real-Time Intelligence. It covers configuring Spark diagnostics, emitting logs and metrics to Azure destinations, and querying data with KQL for effective performance monitoring and diagnostics.⫸ Announcing Microsoft SQL Server 2025: Enterprise AI-ready database from ground to cloud. This article introduces Microsoft SQL Server 2025, an AI-ready database designed for hybrid environments. It highlights built-in AI capabilities, enhanced security and performance features, integration with Microsoft Fabric and Azure Arc, and tools for real-time analytics and developer productivity.⫸ Build Polymorphic Associations in SQL Server with Foreign Keys: This article addresses the challenge of creating polymorphic associations in SQL Server, where a foreign key references multiple tables. It explains the concept, illustrates it with a media review database example, and offers design workarounds to maintain data integrity and simplify schema management.⫸ Manage Azure Data Factory pipelines in Microsoft Fabric: This article explores managing existing Azure Data Factory (ADF) pipelines within Microsoft Fabric, offering a solution for centralizing data operations. It details the steps to "mount" ADF environments in Fabric, allowing seamless management while addressing challenges of migration and feature gaps.⫸ PostgreSQL: Query Optimization for Mere Humans. This article discusses optimizing SQL queries by identifying bottlenecks using the PostgreSQL EXPLAIN and EXPLAIN ANALYZE clauses. It covers interpreting execution plans, understanding query performance issues, and provides tips to enhance database efficiency for better user experience.⚡ Quick Wins: BI Hacks for Instant Impact⫸ HighLevel migrates workloads to Firestore: This article explores how HighLevel, a SaaS platform, improved scalability and performance by migrating to Google Firestore. It highlights Firestore's serverless architecture, real-time capabilities, and role in powering HighLevel's AI solutions, enhancing productivity, reliability, and handling rapid database write surges.⫸ AWS Cost Optimization: This article provides actionable tips for optimizing AWS cloud costs. It highlights strategies like minimizing data transfer costs, identifying underutilized EC2 instances, and using cost-allocation tags to reduce waste, streamline operations, and enhance budget management effectively.⫸ Easily recreate your ADX dashboards as Real-Time Dashboards in Fabric: This article explains how to recreate Azure Data Explorer (ADX) dashboards as Real-Time Dashboards in Microsoft Fabric. It covers the benefits of retaining existing data architecture while leveraging Fabric's advanced features and provides step-by-step guidance for transitioning dashboards seamlessly into the Fabric ecosystem.⫸ Learn the Basics of Well-Structured Data: This article explores data literacy, focusing on understanding, structuring, and using data effectively. It highlights key data traits like volume, history, detail, and consistency, explains well-structured data principles, and offers solutions like splitting and pivoting for improving poorly structured datasets.⫸ Introducing ChatGPT Pro: This article introduces ChatGPT Pro, a $200 monthly plan designed for professionals tackling complex problems. It includes access to advanced AI models, such as o1 pro mode, offering enhanced compute capabilities for improved accuracy and reliability in fields like data science, programming, and research.⫸ Sora is here: This article introduces Sora Turbo, an advanced video generation model by OpenAI, now available to ChatGPT Plus and Pro users. It enables realistic video creation from text, images, and videos, offering enhanced storytelling tools with safety features to ensure responsible use.⫸ Training Language Models on Google Colab: This article provides a guide to fine-tuning Large Language Models on Google Colab without losing progress. It explains using Google Drive to save intermediate results, creating save and load functions for model checkpoints, and ensuring continuity in training across sessions.🎤 Voices of BI: Lessons from Industry Experts⫸ How to Integrate AI and Data Science into Your Business Strategy: This article provides a blueprint for conducting a two-day strategy workshop to integrate AI and machine learning into business strategy. It covers preparation, attendee selection, deep-dive topic identification, and post-workshop actions, offering a versatile, industry-agnostic approach for businesses of any size.⫸ Bridging the Data Literacy Gap. The Advent, Evolution, and Current: This article highlights the evolving role of "Data Translators," professionals bridging the gap between business leaders and data teams to drive data-informed decision-making. It explores challenges like balancing resource abundance with actionable insights and emphasizes the critical need for data literacy to maximize organizational impact.⫸ GPS Interpolation Using Maps and Kinematics: This article explores how to enhance vehicle telematics datasets by interpolating GPS locations between signal changes. It explains packaging approaches, demonstrates challenges with repeated GPS data, and outlines how to use maps and speed signals for accurate geospatial interpolation, improving dataset resolution and value.⫸ Power BI in Teams – ‘Teams activity analytics’ report deprecation: This blog announces the deprecation of Power BI's 'Teams activity analytics' report, effective February 1, 2025, and recommends using the native 'Teams Analytics' feature for comprehensive insights into Teams usage and activities.⫸ From Code to Paper: Using GPT Models and Python to Generate Scientific LaTeX Documents. This blog discusses automating the conversion of algorithms into LaTeX-formatted scientific documents using GPT models. It explores structuring repositories, leveraging GPT for consistency and accuracy, and creating adaptable, professional frameworks for documenting complex algorithms in large projects.⫸ SQL vs. Calculators: Building Champion/Challenger Tests from Scratch. This blog explores the impact of A/B testing (Champion-Challenger testing) on business decision-making, inspired by the famous $300 million button story. It provides a practical guide to implementing this method using Oracle SQL, focusing on hypothesis testing, statistical parameters, and optimizing outcomes like payment rates through controlled experiments.We’ve got more great things coming your way—see you soon!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}} @media only screen and (max-width: 100%;} #pad-desktop {display: none !important;} }

How Generative AI and Governance Help Scale Enterprise Analytics, Automating BI @media only screen and (max-width: 100%;} #pad-desktop {display: none !important;} }👋 Hello ,🦋 Welcome to BIPro #80 – Your Weekly Business Intelligence Boost! 🚀Discover this week’s top BI trends, strategies, and insights to elevate your data-driven success!🚨 Packt Conference Alert! 🚨Stay at the forefront of AI innovation! 🚀 Join us for 3 action-packed days of LIVE sessions with 20+ top experts and unleash the full power of Generative AI at our upcoming conference. Don’t miss out - Claim your spot today!📊 Future-Ready Insights: Data Viz Trends✦ Handling Missing Data in R✦ Data Lakes: Zones and Containers Planning✦ Optimize Spark Compute in Microsoft Fabric✦ Explore Pandas in Python🔄 Transformative Insights: Data in Action✦ Actionable Data Insights for Decision-Making✦ Web Scraping with Python: Scrapy Framework✦ Utilizing VizQL Data Service in Tableau✦ Enhanced Tenant Delegation in Microsoft Fabric⚡ Quick Wins: BI Hacks✦ Visualize Data with Pie Charts in Matplotlib✦ Competitive Edge with AI Strategies✦ Google’s New Generative AI Learning Paths✦ Simplify SQL with Pipe Syntax in BigQuery✦ Shopify’s ML Enhancements for Search Intent🎤 Voices of BI: Expert Insights✦ Fairness in ChatGPT✦ Scaling Analytics: Generative AI and Governance✦ Automating BI: Overcoming Bottlenecks✦ Data Sharing Patterns on AWSGet ready to level-up your business intelligence game! Happy reading!Calling All Data & BI Enthusiasts!Do you dream of sharing your insights and building your reputation in the Data & BI community? Contribute to our new column in the Packt BIPro newsletter! Share your experiences, discuss new BI tools, or ask questions. Gain recognition among 37,000 BI professionals. Reply with your Google Docs article or use our weekly feedback form. Enjoy a free PDF of "Interactive Data Visualization with Python - Second Edition" for participating. Click reply or share your content today!Share your thoughts and opinions here!Cheers,Merlyn ShelleyEditor-in-Chief, PacktPackt’s Signature Series: New Titles Just Arrived!📚➽Learn Microsoft Fabric: Explore Microsoft Fabric's features through real-world examples to build robust data analytics solutions, including lakehouses and data warehouses. Learn to monitor and manage your analytics system for flexibility, performance, and security, while leveraging AI-driven insights with Copilot integration. Start your free trial for access, renewing at $19.99/month.eBook $24.99 $35.99Print + eBook $30.99 $44.99➽Microsoft Power BI Cookbook - Third Edition: Dive into Microsoft Data Fabric to enhance data strategies and gain deeper insights. Effortlessly create Hybrid tables and comprehensive scorecards while utilizing new visualization tools that transform complex data into clear, actionable charts and reports for effective decision-making in Power BI. Start your free trial for access, renewing at $19.99/month.eBook $29.99 $43.99Print + eBook $41.98 $59.99➽Fundamentals of Analytics Engineering: Explore how analytics engineering aligns with your organization's data strategy while gaining insights from seven industry experts. Address common challenges faced by businesses and learn to implement scalable analytics solutions, from data ingestion to visualization, using industry-leading tools. Start your free trial for access, renewing at $19.99/month.eBook $24.99 $35.99Print + eBook $44.99➽Getting Started with DuckDB: Utilize DuckDB to efficiently load, transform, and query diverse data sources and formats. Gain hands-on experience with SQL, Python, and R for data analysis, while exploring how open-source tools and cloud services enhance DuckDB’s versatile capabilities in the data ecosystem. Start your free trial for access, renewing at $19.99/month.eBook $29.99 $43.99Print + eBook $54.99📊 Data Viz Trends Shaping the Future of Insights➽ How to Handle Missing Data in R? This blog explains handling missing data in R, covering data loading, identifying missing values with functions like is.na() and summary(), removing them using na.omit(), and applying imputation methods such as mean, KNN, and multiple imputation for accurate analysis.➽ Data Lake implementation – Data Lake Zones and Containers Planning: This blog discusses Azure Data Lake implementation, focusing on data lake zones, storage accounts, and container planning. It covers raw, enriched, and development data layers, governance, security, and the medallion architecture for effective data organization.➽ Optimizing Spark Compute for Medallion Architectures in Microsoft Fabric: This blog offers guidance on optimizing data engineering workloads using the Medallion architecture, detailing tailored compute configurations for Bronze, Silver, and Gold layers to enhance performance, efficiency, and data accessibility across large-scale datasets.➽ Explore Pandas in Python to Analyze and Manipulate Tabular Data: This blog introduces Pandas, an open-source Python library for data manipulation and analysis. It highlights its key features, installation process, and demonstrates usage through Pandas Series and DataFrames for various data operations and arithmetic calculations.🔄 Real-World Transformation: How Gen BI Made Data Work➽ Enabling Critical Decision-Making with Valuable Data Insights: This blog addresses the challenge of finding quality data for decision-making and introduces the Melissa Data Marketplace, offering accurate, industry-specific data products. It highlights accessibility options and use cases in real estate and healthcare for enhanced data quality.➽ Web Scraping with Python Scrapy Framework: This blog discusses the challenges of manual data collection and introduces web scraping as an efficient solution for automated data extraction. It highlights the Scrapy Python framework, emphasizing its capabilities for structured data gathering and analysis.➽ How to Use VizQL Data Service in Your Tableau Cloud Site? This blog announces the expansion of the VizQL Data Service Developer Preview to all Tableau Cloud customers, highlighting new API Access permissions for enhanced data control, and introducing a Postman Collection for easier API interaction and testing.➽ Announcing the Enhanced Tenant Setting Delegation for Export Controls in Microsoft Fabric: This highlights an enhancement to Microsoft Fabric's Tenant Setting Delegation feature, enabling granular control over data export permissions at the workspace level. It improves security, management, and flexibility for workspace administrators while reducing the burden on tenant admins.⚡ Quick Wins: BI Hacks for Instant Impact➽ Visualization of Data with Pie Charts in Matplotlib: This article explores creating four types of pie charts using a dataset from my Master's Thesis on NIH-funded heart disease research. It emphasizes effective visualization of categorical data with Matplotlib, highlighting insights into gender representation in publications.➽ Carving Out Your Competitive Advantage with AI: This blog discusses how companies can achieve a competitive advantage with AI despite the technology becoming commonplace. It emphasizes creativity in AI applications, the importance of tailored strategies, and the integration of unique datasets and domain expertise.➽ Four new Google’s Gen AI learning paths on offer: This blog addresses the skills gap in AI readiness among organizations and introduces Google Cloud's new generative AI learning paths. These courses aim to equip developers with practical skills to leverage AI effectively, enhancing productivity and career opportunities.➽ Simplify your SQL with pipe syntax in BigQuery and Cloud Logging: This blog introduces SQL pipe syntax, an innovative extension of standard SQL that enhances simplicity and flexibility. It allows for easier data analysis by enabling sequential operator application, improving readability and productivity for users.➽ How Shopify improved consumer search intent with real-time ML? This blog outlines Shopify's integration of AI-powered search capabilities into merchant storefronts, enhancing the shopping experience through Semantic Search and real-time embeddings. This system boosts sales by improving product relevance and search accuracy.🎤 Voices of BI: Lessons from Industry Experts➽ Evaluating fairness in ChatGPT: This blog discusses the careful design of training processes for language models like ChatGPT to minimize harmful outputs and biases. It explores how cues, such as users' names, can influence responses and impact first-person fairness.➽ How Generative AI and Governance Help Scale Enterprise Analytics? This blog summarizes Alteryx's announcements from recent Inspire user conferences, highlighting advancements in Generative AI, the introduction of Alteryx Marketplace, and enhancements to Alteryx Designer and Server, focusing on improved data-driven decision-making and enterprise connectivity.➽ Automating BI: Breaking Down Bottlenecks with Artificial Intelligence: This blog addresses time-to-value challenges in analytics, highlighting IDC research on data decay and underutilization. It emphasizes the need for automation and generative AI to alleviate bottlenecks in the analytics process, enhancing decision-making efficiency.➽ Demystify data sharing and collaboration patterns on AWS: Choosing the right tool for the job. This blog discusses the importance of treating data as a product to overcome challenges like data silos and governance issues. It highlights the benefits of data lakes and the data mesh framework, emphasizing the roles of various personas and AWS services like AWS Glue, AWS Data Exchange, and AWS Clean Rooms for effective data sharing and collaboration.We’ve got more great things coming your way—see you soon!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{line-height:0;font-size:75%} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}} @media only screen and (max-width: 100%;} #pad-desktop {display: none !important;} }

Automated Migration - Alteryx to Microsoft Fabric, OpenAI Realtime API Simplifies Voice Agent Flows @media only screen and (max-width: 100%;} #pad-desktop {display: none !important;} }Transform Your GRC Program. No More Chasing Evidence.If you’re responsible for the GRC program in your organization, don't chase stakeholders for evidence.Use Anecdotes to continuously and indepandantly monitor your tech stack with credible GRC data.Whether you’re complying with SOX, NIST, PCI, or a custom framework, stop managing them in isolation. With Anecdotes’ advanced cross-mapping solution, you can reuse shared evidence across different scopes. Focus on strategy and strengthen your GRC program with Anecdotes.Let’s TalkSponsored🦋 Welcome to BIPro #78 – Your Weekly Business Intelligence Boost! 🚀Dive into this week’s freshest trends, strategies, and insights designed to elevate your data-driven success!📊 Future-Ready Data Visualization Trends✦ Low-Code AI Revolution: Harness Kumologica and Anthropic AI for effortless integration!✦ Search Engine Mastery: Craft powerful algorithms with ClickHouse.✦ LLM Apps Supercharged: Unlock potential using DSPy and Langfuse.✦ Voice Flow Simplified: Discover the ease of the new OpenAI Realtime API.✦ Data Tables Made Easy: Kickstart your Python web apps with robust data solutions.✦ Power BI Highlights: Your guide to the September 2024 features.🔄 Transformative Success Stories in BI✦ Data Management Made Simple: Efficiently delete large data sets in SQL Server.✦ Natural Language Queries: Unlock SQL with user-friendly applications.✦ SQL Server Insights: Navigate stored procedures, functions, and views seamlessly.✦ AI Alignment Unpacked: Explore the Gridworlds problem for innovative solutions.✦ AI Success Formula: Combining Kafka with AI guardrails for optimal performance.✦ Monthly BI Update: Fabric’s latest enhancements for September 2024.⚡ Instant Impact: Quick BI Hacks✦ LLM Integration Simplified: Leverage Scikit-Learn with Scikit-LLM effortlessly.✦ Command-Line Mastery: Build sleek Python apps using Click.✦ AI Chatbot Evolution: Maintain message history with LangChain and SQL.✦ Text Data Transformation: Get AI-ready with no-code solutions.✦ Seamless Integration: Google Cloud Cortex Framework meets Oracle EBS.🎤 BI Voices: Wisdom from Industry Leaders✦ Keynotes to Remember: Highlights from Microsoft and Redgate, plus insights from PASS Data Community Summit.✦ SQL Search Optimization: Master SQL LIKE wildcard searches for better performance.✦ Vector Search Simplified: Zero ETL solutions for Amazon DynamoDB with OpenSearch Service.✦ Alteryx Applications Unveiled: Discover 6 common use cases for impactful data meaning.✦ Streamlined Migration: Transitioning from Alteryx to Microsoft Fabric made easy.Get ready to boost your business intelligence game! Happy reading!Calling All Data & BI Enthusiasts!Do you dream of sharing your insights and building your reputation in the Data & BI community? Contribute to our new column in the Packt BIPro newsletter! Share your experiences, discuss new BI tools, or ask questions. Gain recognition among 37,000 BI professionals. Reply with your Google Docs article or use our weekly feedback form. Enjoy a free PDF of "Interactive Data Visualization with Python - Second Edition" for participating. Click reply or share your content today!Share your thoughts and opinions here!Cheers,Merlyn ShelleyEditor-in-Chief, PacktPackt’s Signature Series: New Titles Just Arrived!📚➽ AI-Assisted Programming for Web and Machine Learning: Unlock the power of AI-assisted programming to streamline web development and machine learning. Learn to enhance frontend and backend coding, optimize ML models, and automate tasks using GitHub Copilot and ChatGPT. Perfect for boosting productivity and refining workflows. Start your free trial for access, renewing at $19.99/month.eBook $18.99 $38.99Print + eBook $32.99 $47.99➽ Machine Learning and Generative AI for Marketing: Leverage AI and Python to revolutionize your marketing strategies with predictive analytics and personalized content creation. Learn to combine advanced segmentation techniques and generative AI to boost customer engagement while ensuring ethical AI practices. Perfect for driving real business growth. Start your free trial for access, renewing at $19.99/month.eBook $19.99 $39.99Print + eBook $34.98 $49.99➽ Amazon DynamoDB - The Definitive Guide: Master Amazon DynamoDB with this comprehensive guide, learning key-value data modeling, optimized strategies for transitioning from RDBMS, and efficient read consistency. Discover advanced techniques like caching and analytics integration with AWS services to boost performance, while minimizing latency and costs. Start your free trial for access, renewing at $19.99/month.eBook $17.99 $35.99Print + eBook $30.99 $44.99➽ Microsoft Power BI Performance Best Practices - Second Edition: Master Power BI performance optimization with this guide, learning to build efficient data models, apply row-level security, and troubleshoot issues using DAX Studio and VertiPaq Analyzer. Implement formal performance management strategies to ensure scalable, high-performing solutions. Start your free trial for access, renewing at $19.99/month.eBook $19.99 $39.99Print + eBook $34.98 $49.99➽ Polars Cookbook: Unlock faster, more efficient data analysis with Python Polars through step-by-step recipes. Master data manipulation, advanced querying, and performance optimization. Learn to handle large datasets, perform complex transformations, and integrate Polars with other tools. Start your free trial for access, renewing at $19.99/month.eBook $17.99 $35.99Print + eBook $30.99 $44.99📊 Data Viz Trends Shaping the Future of Insights➽ Low Code AI Agent Using Kumologica, Anthropic AI: This blog discusses how to use Kumologica and Anthropic AI to create an AI agent for customer feedback analysis in a mobile app. It walks through building an API, analyzing sentiment, and storing results in AWS DynamoDB.➽ Create a Search Engine, Algorithm With ClickHouse: This blog explains how to build a cost-effective, alternative search engine using ClickHouse instead of Elasticsearch. It covers indexing, scoring, and matching search queries with a unified dataset, improving search performance and efficiency.➽ Supercharge Your LLM Apps Using DSPy and Langfuse: This blog explores the rise of large language models (LLMs) and highlights challenges like prompt engineering. It introduces DSPy and Langfuse, frameworks to simplify LLM app development, optimize performance, and enhance debugging through observability and modular design.➽ Exploring How the New OpenAI Realtime API Simplifies Voice Agent Flows: This article reviews OpenAI's new Realtime API, which simplifies building low-latency, speech-to-speech AI applications. It compares previous multi-service voice agent workflows with the streamlined Realtime API setup, showcasing implementation, cost analysis, and potential benefits.➽ Getting Started with Powerful Data Tables in Your Python Web Apps: This blog details how a Python developer can create an interactive, feature-rich data grid using the Reflex framework and AG Grid without needing JavaScript. It explains building a finance app to display and manipulate stock data, with features like sorting, filtering, and graphing.➽ Power BI September 2024 Feature Summary: This post highlights Power BI's new features, including the much-anticipated Dark Mode, default cross-page summaries in Copilot, a streamlined menu bar, and Metrics Hub for consistent data management. It also introduces updates for visual calculations and formatting options.🔄 Real-World Transformation: How Gen BI Made Data Work➽ How to Delete Large Amounts of Data in Microsoft SQL Server? This blog discusses efficient techniques for large-scale data deletion in Microsoft SQL Server, including batching with the DELETE command, using TRUNCATE, partition switching, SELECT INTO, disabling indexes, and applying table locks. It emphasizes best practices like monitoring log growth and testing in non-production environments.➽ Natural Language SQL Query Application: This blog details building a web app that converts natural language into SQL queries using React, Node.js, PostgreSQL, and OpenAI. It simplifies querying for non-technical users, enabling seamless database interactions with natural language inputs, improving data accessibility.➽ SQL Server Metadata for Stored Procedures, Functions and Views: This blog demonstrates how to create, use, and track SQL modules like user-defined functions, stored procedures, and views in SQL Server. It also covers best practices for managing these modules using T-SQL scripts and metadata queries for efficient database management.➽ Exploring the AI Alignment Problem with Gridworlds: The blog discusses the AI alignment problem, highlighting risks of advanced AI misaligning with human interests. It critiques common objections, explores hidden objectives in AI learning, and introduces "AI Safety Gridworlds" for testing AI behavior without explicit instructions.➽ How to succeed with AI: Combining Kafka and AI Guardrails? This article explores the intersection of AI and Kafka, emphasizing the necessity of AI guardrails to mitigate risks like data leaks and bias. It argues that effective AI relies on real-time data streaming and robust governance for optimal performance.➽ Fabric September 2024 Monthly Update: This post highlights exciting updates for FabCon Europe, including Copilot integration in Dataflows Gen2 and Power BI, enhanced Git functionality, a redesigned Real-Time hub, and new features in Data Engineering and Data Science for improved AI data management and collaboration.⚡ Quick Wins: BI Hacks for Instant Impact➽ Integrating LLMs with Scikit-Learn Using Scikit-LLM: This post introduces the Scikit-LLM library, bridging Scikit-Learn and large language models for enhanced text classification. It details installation, backend support, and implementation of a zero-shot text classifier on a sentiment analysis dataset, showcasing improved performance.➽ Building Command Line Apps in Python with Click: This blog discusses the Click library for Python, which simplifies the creation of command-line applications. It covers features like easy command composition, integration with other libraries, and provides examples for building a file organizer and calculating rectangle areas.➽ AI Chatbot with Message History using LangChain and SQL: This blog provides a tutorial on enhancing LLM applications by adding message history and a user interface using LangChain. It guides readers through building a Flask chatbot, integrating local memory, and using prompt templates for better interactions.➽ Making Text Data AI-Ready. An introduction using no-code solutions: This blog explains how to make unstructured text data AI-ready for large language models (LLMs), outlining the importance of formatting, specifically using Markdown, and providing no-code tools like Jina AI and LlamaParse for efficient text processing.➽ Google Cloud Cortex Framework integrated with Oracle EBS: This blog discusses the importance of rapid data access for businesses, highlighting the integration of Oracle E-Business Suite with Google Cloud’s Cortex Framework to enhance data visibility, improve order-to-cash processes, and facilitate actionable insights.🎤 Voices of BI: Lessons from Industry Experts➽ Presenting the Microsoft Keynote, Redgate Keynote, and a community star - PASS Data Community Summit: The PASS Summit 2024 is a premier event for data professionals, featuring three keynotes from industry leaders on AI innovation with Azure Databases, practical Database DevOps solutions, and leveraging AI to enhance productivity. Attendees can explore various learning pathways and engage in valuable networking opportunities.➽ Optimize SQL LIKE Wildcard Searches: This blog addresses the inefficiencies of full wildcard searches using SQL's LIKE operator in Microsoft SQL Server. It explores optimization techniques, including binary collation and Full-Text Search, to enhance query performance and minimize execution time significantly.➽ Vector search for Amazon DynamoDB with zero ETL for Amazon OpenSearch Service: This blog explains how to integrate Amazon DynamoDB with Amazon OpenSearch Service and Amazon Bedrock for advanced data insights and generative AI capabilities. It covers setting up zero-ETL integration, generating embeddings, and enhancing search functionalities through practical examples.➽ What is Alteryx Used For: 6 Common Use Cases. This blog introduces Alteryx, an analytics automation platform that streamlines data collection, preparation, and blending to provide actionable insights. It highlights its applications in data analytics, predictive modeling, and geospatial analysis, offering consultation services for businesses to optimize their data processes.➽ Automated Migration - Alteryx To Microsoft Fabric Conversion: This blog discusses the challenges and considerations involved in migrating workflows from Alteryx to Microsoft Fabric. It highlights differences in functionality, data integration, workflow complexity, and advanced analytics, providing insights to facilitate a successful migration process.We’ve got more great things coming your way—see you soon!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{line-height:0;font-size:75%} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}} @media only screen and (max-width: 100%;} #pad-desktop {display: none !important;} }

Data Governance in Data Science Pipelines, Alteryx AI & Analytics Automation for Business Success @media only screen and (max-width: 100%;} #pad-desktop {display: none !important;} }👋 Hello ,🦋 Welcome to BIPro #77 – Your Weekly Business Intelligence Boost! 🚀Get ready for this week’s latest BI trends, strategies, and insights to fuel your data-driven success!📊 Trending Now: The Future of Data Visualization✦ Harness the Power of Dataflow: 5 solution guides for common Dataflow use cases.✦ Master Data Governance: Simplify governance with AWS Lake Formation & IAM Identity Center.✦ Data Science Best Practices: Implement governance techniques in your data pipelines.✦ Alteryx for All: How Alteryx makes data analytics accessible to everyone.🔄 Real-World Transformations: Data in Action✦ BigQuery Gets Smarter: Vector search goes GA – here’s what it means for you.✦ TimeGPT Takes the Lead: Forecast stock markets with cutting-edge TimeGPT.✦ Inside Transformers: Visualize model internals with Hugging Face.✦ Marketing Mastery: Python-powered Marketing Mix Modeling.⚡ BI Hacks for Instant Wins✦ Gemini in Looker: Supercharge LookML & visualizations with AI assistance.✦ Google Workspace Insights: The Analytics Block now in Looker Marketplace.✦ AI Chatbots Made Easy: Build one with message history using LangChain & SQL.✦ Marketing Automation: 4 ways marketing leaders succeed with Alteryx AI.🎤 Expert Insights: Voices of BI Leaders✦ Generative AI Power: Fuel your data with AI for game-changing insights.✦ Azure Data Studio Tips: Master the Import Extension for streamlined data workflows.✦ Code Upgrade: Convert old running total code to efficient window functions.Enjoy your BI power-up this week! 🎉Calling All Data & BI Enthusiasts!Do you dream of sharing your insights and building your reputation in the Data & BI community? Contribute to our new column in the Packt BIPro newsletter! Share your experiences, discuss new BI tools, or ask questions. Gain recognition among 37,000 BI professionals. Reply with your Google Docs article or use our weekly feedback form. Enjoy a free PDF of "Interactive Data Visualization with Python - Second Edition" for participating. Click reply or share your content today!Share your thoughts and opinions here!Cheers,Merlyn ShelleyEditor-in-Chief, PacktPackt’s Signature Series: New Titles Just Arrived!📚➽ AI-Assisted Programming for Web and Machine Learning: Unlock the power of AI-assisted programming to streamline web development and machine learning. Learn to enhance frontend and backend coding, optimize ML models, and automate tasks using GitHub Copilot and ChatGPT. Perfect for boosting productivity and refining workflows. Start your free trial for access, renewing at $19.99/month.eBook $18.99 $38.99Print + eBook $32.99 $47.99➽ Machine Learning and Generative AI for Marketing: Leverage AI and Python to revolutionize your marketing strategies with predictive analytics and personalized content creation. Learn to combine advanced segmentation techniques and generative AI to boost customer engagement while ensuring ethical AI practices. Perfect for driving real business growth. Start your free trial for access, renewing at $19.99/month.eBook $19.99 $39.99Print + eBook $34.98 $49.99➽ Amazon DynamoDB - The Definitive Guide: Master Amazon DynamoDB with this comprehensive guide, learning key-value data modeling, optimized strategies for transitioning from RDBMS, and efficient read consistency. Discover advanced techniques like caching and analytics integration with AWS services to boost performance, while minimizing latency and costs. Start your free trial for access, renewing at $19.99/month.eBook $17.99 $35.99Print + eBook $30.99 $44.99➽ Microsoft Power BI Performance Best Practices - Second Edition: Master Power BI performance optimization with this guide, learning to build efficient data models, apply row-level security, and troubleshoot issues using DAX Studio and VertiPaq Analyzer. Implement formal performance management strategies to ensure scalable, high-performing solutions. Start your free trial for access, renewing at $19.99/month.eBook $19.99 $39.99Print + eBook $34.98 $49.99➽ Polars Cookbook: Unlock faster, more efficient data analysis with Python Polars through step-by-step recipes. Master data manipulation, advanced querying, and performance optimization. Learn to handle large datasets, perform complex transformations, and integrate Polars with other tools. Start your free trial for access, renewing at $19.99/month.eBook $17.99 $35.99Print + eBook $30.99 $44.99➽ 15 Math Concepts Every Data Scientist Should Know: Master key data science algorithms through Python-based examples, boosting your solutions by applying and creating algorithms. Learn foundational and advanced mathematical techniques for solving real-world data challenges, with practical Python applications. Start your free trial for access, renewing at $19.99/month.eBook $17.99 $35.99Print + eBook $30.99 $44.99📊 Data Viz Trends Shaping the Future of Insights➽ Five solution guides for common Dataflow use cases: This article introduces Dataflow solution architectures for real-time data processing, offering practical guides for use cases like machine learning, ETL, log replication, marketing intelligence, and clickstream analytics, highlighting Dataflow's scalability, flexibility, and AI integration capabilities.➽ Apply enterprise data governance and management using AWS Lake Formation and AWS IAM Identity Center: This article discusses a solution using AWS Lake Formation and IAM Identity Center to address challenges in managing and governing legacy data during digital transformation. It outlines strategies for preserving historical data, enforcing compliance, and maintaining secure, role-based access, enabling seamless transitions without altering existing user entitlements.➽ Implementing Data Governance in Data Science Pipelines: Techniques and Best Practices. This article explores key techniques and best practices for implementing data governance in data science pipelines. It emphasizes data quality, regulatory compliance, and risk management while outlining processes like role definition, metadata management, quality assurance, and auditing to ensure secure, efficient, and traceable data usage.➽ How Alteryx Makes Data Analytics Accessible to Everyone? This blog highlights how Alteryx simplifies data analytics by making it accessible to users of all skill levels. It explains Alteryx's key features, including drag-and-drop workflows, automated data preparation, and advanced analytics tools, enabling non-technical users to efficiently analyze data and make data-driven decisions across various industries.🔄 Real-World Transformation: How Gen BI Made Data Work➽ BigQuery vector search is now GA: This article announces the general availability of BigQuery vector search, enabling vector similarity search on data stored in BigQuery. It enhances data analytics by using AI models to encode semantic meaning as vector embeddings, empowering applications like semantic search, anomaly detection, and drug discovery with improved scalability and performance.➽ Stock Market Forecasting with TimeGPT: This article introduces TimeGPT, a Transformer-based model designed for time series forecasting. It explains how to use TimeGPT via the Nixtla API for both simple and advanced forecasting techniques, including stock market predictions, with minimal code and high performance.➽ How to Visualize Model Internals and Attention in Hugging Face Transformers? This tutorial explains how to visualize the internal workings and attention mechanisms of Hugging Face Transformer models. It demonstrates techniques such as gradient-based visualization, attention heatmaps, and hidden state analysis to help users better understand model predictions and attention distribution in sentences.➽ Mastering Marketing Mix Modelling In Python: This series provides a hands-on guide to mastering Marketing Mix Modeling (MMM) using the pymc-marketing Python package. It covers key topics such as model training, validation, Bayesian priors, and budget optimization, offering practical tools to enhance marketing strategies through Bayesian MMM.⚡ Quick Wins: BI Hacks for Instant Impact➽ Gemini in Looker LookML Assistant and Visualization Assistant: This article introduces two new AI-driven features in Looker, LookML Assistant and Visualization Assistant, powered by Google’s Gemini. These tools simplify creating and customizing data models and visualizations using natural language, accelerating business intelligence workflows. They enhance collaboration and decision-making by making data insights more accessible and customizable across organizations.➽ Bringing Google Workspace Analytics Block to Looker Marketplace: The Google Workspace Analytics Block in Looker Marketplace offers pre-built metrics for Workspace administrators to track adoption, collaboration, and security. It enables customized dashboards, automated reporting, and integrates with existing workflows, empowering IT admins and business leaders to make data-driven decisions and enhance productivity.➽ AI Chatbot with Message History using LangChain and SQL: This article provides a guide for adding message history and a user interface (UI) to an LLM-based application. It explains how to use LangChain, Flask, and SQLite to create a chatbot with message history, leveraging prompt templates and the RunnableWithMessageHistory class for managing conversations in a deployable AI application.➽ Top 4 Ways on How Marketing Leaders Use Alteryx AI & Analytics Automation for Business Success: This blog explores four ways marketing leaders use Alteryx AI and analytics automation to streamline data management, enhance campaigns, optimize resources, and achieve immediate results. It highlights how Alteryx helps centralize data, predict trends, automate tasks, and integrate future-ready solutions.🎤 Voices of BI: Lessons from Industry Experts➽ Fuel Your Data with Generative AI: This article highlights how generative AI can enhance data management, focusing on three use cases: automating data integration (ETL), enabling conversational business intelligence, and generating synthetic data for testing and innovation. It emphasizes AI’s role in unlocking data’s potential, improving accessibility, and accelerating insights across organizations.➽ Diving Deeper into the Import Extension in Azure Data Studio: This article explores using the Import extension in Azure Data Studio (ADS) to handle complex imports, including derived columns, number manipulations, and data masking. The author demonstrates how to create derived columns, apply transformations, and experiment with importing data from various sources, highlighting successes and challenges with calculations and data formatting during the import process.➽ Converting Old Running Total Code to Window Functions: This article explores optimizing a running total calculation of the previous five rows using SQL Server's window functions. It compares the original solution, which used cross joins and left joins, to a more efficient approach with the SUM function and an OVER clause. The article also demonstrates testing and refactoring processes, highlighting the use of window functions for improved performance.We’ve got more great things coming your way—see you soon!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{line-height:0;font-size:75%} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}} @media only screen and (max-width: 100%;} #pad-desktop {display: none !important;} }

Colossus, Google’s not-so-secret storage engine | Doris vs ElasticsearchHow to balance cloud agility, cost, and riskJoin cybersecurity thought leader David Linthicum for a special fireside chat to learn how to use AI and ML to unify your data strategies, uncover hidden cloud costs, and overcome the limitations of your traditional data protection in public cloud environments.Save Your SpotSponsoredSubscribe | Submit a tip | Advertise with us📬BIPro#96~ your trusted signal through the BI noise.This week, we zoom in on how data teams are evolving: from the tools they use to the decisions about who should own them. As gen AI matures and enterprise data landscapes become more fragmented, the value of thoughtful orchestration and governance is more vital than ever.Here’s what’s sparking conversations in this issue:🔍 From Spreadsheets to Smart AgentsBuild your own AI coding assistant with Ollama and Hugging Face inside JupyterLab, no cloud required.Google’s Data Science Agent gets tested in the real world; can it really replace a data analyst?🧠 Smarter, Cleaner Data Starts HereGet hands-on with 10 Pandas One-Liners to clean up messy datasets fast.Dive deep into BigQuery’s new Gemini-powered prep tools, now GA.Learn how SQL Server’s new fuzzy search functions simplify approximate matching.📈 BI Teams, Tools, and TradeoffsWho should own BI? IT ensures control, but business drives speed, find out why a hybrid model may be the future.Follow Prime Video’s dashboarding overhaul with Amazon QuickSight: better governance, lower cost, happier teams.🧰 Gen AI Meets Real-World InfrastructureDiscover how agents connect to Google Cloud databases securely and in real-time.Understand AtScale’s Universal Semantic Layer, a game-changer for unified logic across BI tools.Explore Colossus, Google’s not-so-secret storage engine delivering SSD performance at HDD prices.⚡ Quick Wins & Industry VoicesCapital on Tap’s case study on data masking at scale using DataVeil.Doris vs Elasticsearch, who wins on cost, speed, and scalability for real-time analytics?And a new entry from Melissa on Snowflake Marketplace for instant data quality and enrichment.Whether you're an engineer digging deep into data pipelines or a decision-maker chasing clarity, this issue gives you the sharpest tools, honest evaluations, and stories from the trenches.Let’s sharpen your week with insights that matter.Merlyn ShelleyGrowth Lead, Packt📊 Data Viz Trends Shaping the Future of Insights10 Pandas One-Liners for Data Cleaning: This article presents 10 concise pandas one-liners to clean messy datasets, tackling missing values, formatting errors, outliers, and inconsistent categories. From standardizing text and email formats to handling duplicates and validating data, these quick fixes simplify real-world data preparation using minimal code.Understanding Database Consistency: This article explains database consistency models in distributed systems, including strong, eventual, causal, monotonic, and read-your-writes consistency. It covers their practical applications, trade-offs with availability and partition tolerance, and guides readers in choosing the right model for different real-world scenarios.The future of dashboarding: Prime Video’s migration journey to Amazon QuickSight: Prime Video transformed its business intelligence by migrating from legacy BI tools to Amazon QuickSight. This shift improved performance, reduced costs, and enhanced data governance. Over two years, the team adopted a phased approach, enabling better scalability, automation, and faster decision-making across global teams.AI-assisted BigQuery data preparation now GA: Gartner notes up to 94% of time in complex industries is spent preparing data. BigQuery data preparation, now generally available, uses Gemini to simplify and automate data wrangling. With visual pipelines, low-code tools, and Git integration, teams streamline transformations, ensure quality, and accelerate analytics workflows efficiently.📈 Dive into Databases: SQL EssentialsA Guide to Integrating ChatGPT with Google Sheets: This guide outlines how to integrate ChatGPT with Google Sheets using the GPT for Sheets add-on. It walks through installation, API setup, and practical use cases, from generating content to analyzing data, empowering users to automate tasks, personalize content, and streamline spreadsheet workflows using AI.Doris vs Elasticsearch: A Comparison and Cost Case Study. This article compares Apache Doris and Elasticsearch for real-time analytics and search. Doris excels in complex queries, SQL support, and cost efficiency, while Elasticsearch leads in full-text search. A Tencent Music case study shows Doris reduced storage by 70% and boosted performance, making it a strong alternative for scalable analytics.Accelerate operational analytics with Amazon Q Developer in Amazon OpenSearch Service: Amazon Q Developer now integrates with Amazon OpenSearch Service, allowing users to explore and visualize operational data using natural language. It simplifies alert investigation, speeds up incident resolution, and supports AI-generated summaries, anomaly detection, and dashboard creation, making observability more accessible and reducing time spent on manual troubleshooting.🔄 Real-World Transformation: How Gen BI Made Data WorkImplementing Fuzzy Search in SQL Server Using New Inbuilt Functions: Microsoft SQL Server now supports built-in fuzzy search functions like EDIT_DISTANCE and JARO_WINKLER_SIMILARITY, enabling developers to handle name variations and typos directly within T-SQL. These functions improve search accuracy, reduce external tool reliance, and simplify approximate matching across large datasets, especially useful for user-facing or record-matching applications.Google’s Data Science Agent: Can It Really Do Your Job? Google’s Data Science Agent, now built into Colab, automates data workflows from EDA to model building using natural language prompts. While it speeds up analysis and corrects errors on the fly, it struggles with iterative edits and nuanced decision-making. It’s a helpful tool, but not yet a full data scientist replacement.How Colossus optimizes data placement for performance: Google’s Colossus storage system powers services like Gmail, YouTube, and BigQuery, offering SSD-like speed at HDD costs. With innovations like L4-based SSD caching and writeback, Colossus dynamically places hot data on SSDs. This adaptive approach boosts IOPS and throughput while minimizing costs, supporting massive scale without user-side complexity.⚡ Quick Wins: BI Hacks for Instant ImpactBuild Your Own AI Coding Assistant in JupyterLab with Ollama and Hugging Face: This guide walks through building a private AI coding assistant in JupyterLab using Jupyter AI, Ollama, and Hugging Face. It enables offline coding support, including error fixing, autocompletion, and code generation. Running models locally boosts privacy and responsiveness, ideal for developers seeking control without relying on the cloud.Capital on Tap Meeting Regulatory Compliance and Explosive Growth with DataVeil Data Masking: Capital on Tap used DataVeil to protect sensitive data and meet privacy laws like GDPR and ISO 27001. With 60 databases and fast growth, they needed a way to mask data for testing without exposing real information. DataVeil offered automation, consistency, and ease of use, saving time and keeping them compliant.Who Should Own the Business Intelligence Team - IT or Business? Should the BI team report to IT or the business? IT offers strong governance and technical expertise, while business-led teams move faster and deliver more relevant insights. The best approach is a mix: a central BI team ensures standards and data quality, while business teams focus on their specific needs.🎤 Voices of BI: Lessons from Industry ExpertsUnlock Instant Data Quality and Data Enrichment on Snowflake Marketplace: Snowflake Marketplace now offers instant access to Melissa’s 23 data products, tools and datasets that help clean, verify, and enrich customer data directly in Snowflake. With no complex setup required, businesses can quickly improve data quality, reduce fraud, and drive better decisions through native apps for email, phone, address, and demographic verification.Unified, Cost-Effective Text-to-SQL and Business Intelligence with the AtScale Semantic Layer: AtScale’s Universal Semantic Layer helps organizations deliver consistent, cost-effective data access across tools like Power BI, Excel, and Text-to-SQL platforms. By standardizing business logic across diverse data sources, it eliminates duplicated metrics, reduces data silos, and improves performance, without needing new ETL pipelines. This approach ensures accurate, real-time insights for both technical and business users.Learn how to connect agents to Google Cloud databases: Google Cloud now offers tools to build advanced AI agents that connect directly to databases for real-time, secure data access. With the open-source Gen AI Toolbox for Databases, developers can streamline connections to Google Cloud and open-source databases. This enables agents to query data using natural language, handle complex workflows, and work across graph, vector, and text data models, helping enterprises create smarter, scalable gen AI applications.*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Share your insights in a 30-min interview and choose any ebook from the Packt library!Claim a Free Packt Credit for a Quick 30-Min Interview!Hi ,At Packt, we are always looking for ways to better support data professionals like you in your learning journey.Your input can help us shape future content to better meet your needs.We would love to invite you to a quick 30-minute user insight interview where we can hear about your learning preferences and how we can improve our offerings. ❯❯❯❯ Claim Your Interview Slot!Since you’ve engaged with our data books and newsletters, your perspective would be incredibly valuable in guiding the future of our content.As a token of our appreciation, you'll receive a Packt credit to redeem for any ebook of your choice after the interview.If you're interested, please share your availability here:👉 Reserve Your Interview Slot - it’ll only take 2 - minutes!Thank you for considering, and we look forward to chatting with you!Schedule Your 30-Min SessionCheers,Packt.*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Real-Time Data with Amazon Kinesis, Cloud Storage Data Discovery with Dataplex, Smoothing Data Spike @media only screen and (max-width: 100%;} #pad-desktop {display: none !important;} }The top ten nastiest vulnerabilities of Q3Are you exposed? Download the Q3 2024 Vulnerability Watch report to find out. The usual vulns from Microsoft and VMware make the list, but there are some surprises too. Chances are at least one of these vulnerabilities is lurking in your environment.The report outlines exposure risk specifications and offers practical mitigation actions for each CVE included to reduce your cyber risk. Download the report and stay one step ahead of the most-critical exposure risk.Download Now!Sponsored🗞️Welcome to BIPro #83 – Your Weekly Business Intelligence Kickstart! 🚀Get ready to dive into this week's most exciting BI trends, strategies, and tips to drive your data-forward success!📊 Visualize the Future: Trends and Tips◘ Keep It Clean, Keep It Fast: How index management can boost your database speed.◘ PostgreSQL + Docker, Simplified: A step-by-step setup guide.◘ Embedding Azure Logic Apps: Power up your metadata-driven data platforms.◘ What’s Lurking in Your Dev Database? Hint: Production data.◘ NL2SQL with BigQuery and Gemini: Enhancing SQL with natural language.◘ Copilot in Power BI Mobile: New features (Preview).🔄 Transformations in Action: Real-World Success◘ Streaming with Apache Kafka & Zookeeper: Building a robust data flow.◘ Microsoft Fabric + GraphQL: CRUD operations made easy.◘ New DBA Checklist: Getting up to speed with SQL Server.◘ Data Quality Visualization: Power BI tips for data profiling.◘ Cloud Storage Data Discovery with Dataplex: Effortless cataloging.◘ Upsert & Overwrite Made Easy: Streamlining data ingestion.⚡ Quick Wins: Hacks for Instant BI Impact◘ MySQL Admin Tasks on Azure: Essentials for flexible servers.◘ Marketing Models in Python: Tips to calibrate your approach.◘ AdaBoost Classifier: Get to know this popular model.◘ 4 Pillars of a Data Career: What to focus on for growth.◘ Real-Time Data with Amazon Kinesis: Delivering to OpenSearch.◘ SQL to Fabric Migration: Simple steps for a smooth transition.🎤 Voices of BI: Insights from Industry Pros◘ Boosting Performance in PySpark: Optimization techniques.◘ Smoothing Data Spikes in Python: A guide for Raman spectra.◘ Customer Journeys with Deep Learning: Optimizing experiences.◘ Least Squares Regression Explained: The basics and beyond.◘ Big Data Migration by Delhivery: Moving 500TB with Amazon S3.Get ready to level-up your business intelligence game! Happy reading!Calling All Data & BI Enthusiasts!Do you dream of sharing your insights and building your reputation in the Data & BI community? Contribute to our new column in the Packt BIPro newsletter! Share your experiences, discuss new BI tools, or ask questions. Gain recognition among 37,000 BI professionals. Reply with your Google Docs article or use our weekly feedback form. Enjoy a free PDF of "Interactive Data Visualization with Python - Second Edition" for participating. Click reply or share your content today!Share your thoughts and opinions here!Cheers,Merlyn ShelleyEditor-in-Chief, PacktPackt’s Signature Series: New Titles Just Arrived!📚➽Learn Microsoft Fabric: Explore Microsoft Fabric's features through real-world examples to build robust data analytics solutions, including lakehouses and data warehouses. Learn to monitor and manage your analytics system for flexibility, performance, and security, while leveraging AI-driven insights with Copilot integration. Start your free trial for access, renewing at $19.99/month.eBook $24.99 $35.99Print + eBook $44.99➽Microsoft Power BI Cookbook - Third Edition: Dive into Microsoft Data Fabric to enhance data strategies and gain deeper insights. Effortlessly create Hybrid tables and comprehensive scorecards while utilizing new visualization tools that transform complex data into clear, actionable charts and reports for effective decision-making in Power BI. Start your free trial for access, renewing at $19.99/month.eBook $29.99 $43.99Print + eBook $59.99➽Fundamentals of Analytics Engineering: Explore how analytics engineering aligns with your organization's data strategy while gaining insights from seven industry experts. Address common challenges faced by businesses and learn to implement scalable analytics solutions, from data ingestion to visualization, using industry-leading tools. Start your free trial for access, renewing at $19.99/month.eBook $24.99 $35.99Print + eBook $44.99➽Getting Started with DuckDB: Utilize DuckDB to efficiently load, transform, and query diverse data sources and formats. Gain hands-on experience with SQL, Python, and R for data analysis, while exploring how open-source tools and cloud services enhance DuckDB’s versatile capabilities in the data ecosystem. Start your free trial for access, renewing at $19.99/month.eBook $29.99 $43.99Print + eBook $54.99📊 Data Viz Trends Shaping the Future of Insights⫸ A Tidy Database is a Fast Database: Why Index Management Matters: This blog explores common indexing issues in SQL databases that can degrade performance and increase costs. It covers overlooked, duplicate, fragmented, and missing indexes, offering strategies for effective indexing to optimize database efficiency.⫸ Step by step guide to setup PostgreSQL on Docker: This blog offers a step-by-step guide to installing PostgreSQL on a Mac using Docker, covering prerequisites, setup, volume creation, and container management to simplify PostgreSQL learning and development without overloading system resources.⫸ How To Embed Your Azure Logic Apps in a Metadata-driven Data Platform: This article explains how to streamline Azure Logic Apps for bulk data extraction from multiple SharePoint Lists into Azure SQL, using a metadata-driven framework for efficient, parameterized workflows, minimizing repetitive tasks and enhancing productivity.⫸ What's In Your Development Database? The Answer: Production Data. This article discusses how many development teams still use unmasked production data, revealing privacy concerns and challenges. It examines synthetic data and data-sanitization tools, highlighting their trade-offs in creating realistic data distributions, as well as ongoing issues with data masking and management.⫸ NL2SQL with BigQuery and Gemini: This blog explores Natural Language to SQL (NL2SQL), a technology enabling non-technical users to query databases using plain language. It covers NL2SQL’s transformative potential in democratizing data access, real-world challenges in data quality, and best practices for implementing NL2SQL solutions on Google Cloud.⫸ Introducing Copilot in Power BI Mobile Apps (Preview): This blog introduces you to Copilot in Power BI Mobile apps, an AI-powered feature designed to give you instant report summaries and insights. With Copilot, you can quickly access essential data, make informed decisions, and explore interactive visuals effortlessly.🔄 Real-World Transformation: How Gen BI Made Data Work⫸ Build a Streaming Data Architecture with Apache Kafka and Zookeeper: This article addresses the challenge of capturing and migrating massive real-time data efficiently, showcasing a project-based approach using Apache Kafka and Zookeeper. It provides step-by-step guidance for streaming data from producers to Kafka, with consumer scripts sending data to Elasticsearch and Azure Data Lake Gen 2 for analysis.⫸ CRUD Operations in Microsoft Fabric using GraphQL API Mutations: This article explores using Microsoft Fabric’s GraphQL API to not only query but also modify data through mutations, enabling CRUD operations within a Fabric warehouse. It provides a sample table setup, demonstrates creating a GraphQL API, and explains using mutations for data updates.⫸ Preparing a New DBA to Take Over a SQL Server Environment: This article details a DBA’s process for transitioning their SQL Server management role before retirement. It covers documenting key server information, maintenance jobs, and platform-specific notes, as well as conducting a thorough handover with a new DBA through collaborative review sessions, Q&A meetings, and practical issue-handling experiences. Key takeaways emphasize focused knowledge transfer, effective documentation, and sticking to core responsibilities.⫸ Power BI to Visualize and Profile Data for Data Quality: This blog guides readers on using Power BI to visualize SQL Server data profiling results, addressing common data quality issues and enhancing data analysis by making profiling outputs more accessible and interpretable.⫸ Dataplex discovers and catalogs Cloud Storage data: This article introduces Google Cloud’s Dataplex feature for automatic discovery and cataloging of Cloud Storage data. It highlights how Dataplex scans, classifies, and integrates data into BigQuery for enhanced visibility, reduced manual effort, and accelerated AI and analytics workflows.⫸ Simplifying Data Ingestion with Copy Job: Upsert to SQL Database & Overwrite to Fabric Lakehouse: This article introduces Microsoft Fabric's Copy Job, a tool simplifying data ingestion across sources and destinations with customizable options for data movement. It supports incremental upserts for SQL databases and overwrite capabilities for Fabric Lakehouse tables, enabling flexible data syncing.⚡ Quick Wins: BI Hacks for Instant Impact⫸ Azure Database for MySQL Flexible Server Administrative Tasks: This article covers essential backup operations for Azure Database for MySQL flexible servers, explaining automated and on-demand backups, retention settings, encryption, and recovery options to support business continuity and data protection.⫸ Calibrating Marketing Mix Models In Python: This series on marketing mix modeling (MMM) guides readers in mastering MMM with a focus on model training, validation, calibration, and budget optimization using Python’s pymc-marketing package, helping refine marketing strategies and improve ROI.⫸ AdaBoost Classifier: This article introduces AdaBoost, an adaptive machine learning algorithm that iteratively builds simple decision trees, focusing on correcting previous misclassifications. Using the classic golf dataset, it demonstrates how AdaBoost combines weak learners into a powerful classifier for improved accuracy.⫸ The Four Pillars of a Data Career: If you’re an aspiring data professional, this article guides you through four essential skills: Excel for data manipulation, SQL for querying, visualization tools like Tableau or Power BI for insights, and Python or R for scripting—crucial for landing that first analyst role.⫸ Use Amazon Kinesis Data Streams to deliver real-time data to Amazon OpenSearch Service domains with Amazon OpenSearch Ingestion: This article shows you how to use Amazon Kinesis Data Streams to buffer and aggregate real-time data for Amazon OpenSearch Service. It highlights ways to centralize log aggregation for compliance, scalability, and resilience, streamlining real-time analytics with minimal effort.⫸ SQL to Microsoft Fabric Migration: Beginner-Friendly Strategies for a Smooth Transition. This post covers strategies for integrating SQL Server with Microsoft Fabric to enable seamless analytics and reporting in Power BI. It explores migration techniques, such as Notebooks, Pipelines, and Copy Assistant, for flexible, scalable data movement and incremental updates.🎤 Voices of BI: Lessons from Industry Experts⫸ Optimizing the Data Processing Performance in PySpark: This article explores optimizing PySpark performance on Databricks for large-scale data processing, using a retail transaction dataset as a case study. It highlights common bottlenecks and provides strategies for efficient data handling, feature engineering, and workflow tuning.⫸ Removing Spikes from Raman Spectra with Python: A Step-by-Step Guide. This tutorial offers a Python-based approach for removing cosmic ray-induced spikes from Raman spectra, focusing on key steps like peak finding, spike detection, and spectrum correction to improve data accuracy for spectral analysis.⫸ Data-Driven Journey Optimization: Using Deep Learning to Design Customer Journeys: This post explores combining deep learning and optimization to design high-converting customer journeys. Using LSTM models for predictive journey analysis and beam search for sequence optimization, it addresses limitations in traditional marketing attribution by accounting for touchpoint order, timing, and contextual factors.⫸ Least Squares Regression: This article introduces linear regression fundamentals, focusing on Ordinary Least Squares (OLS) and Ridge regression. It explains how Ridge regression improves model stability by addressing feature sensitivity, illustrated through a sample dataset predicting golfer attendance based on weather conditions.⫸ How Delhivery migrated 500 TB of data across AWS Regions using Amazon S3 Replication: This post walks you through how Delhivery, a leading logistics provider in India, successfully migrated over 500 TB of data to meet Indian data residency laws using Amazon S3 Replication and S3 Batch Operations. You’ll discover their strategies, challenges, and approaches, including near real-time replication to keep data synchronized across AWS Regions while ensuring uninterrupted service for their systems.We’ve got more great things coming your way—see you soon!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}} @media only screen and (max-width: 100%;} #pad-desktop {display: none !important;} }

Identify Anti-Patterns in SQL Server Queries, Attribute-Level Governance Using Apache Iceberg TablesConcerned About AI Mistakes? Learn How to Mitigate the Risks – Read Now.Sponsored🗞️Welcome tothis week’s edition ofBIPro #94, where we bring you the most exciting advancements shaping business intelligence, analytics, and AI.From fully automated data cleaning to streamlined data pipelines and cutting-edge AI innovations, this curated list covers everything you need to stay ahead in the fast-moving world of data.🔍 In This Edition:✅ Automate messy data cleaning with Python to save time and boost accuracy✅ Avoid common Power BI pitfalls for scalable, high-performance dashboards✅ Supercharge SQL Server queries with anti-pattern detection and optimization✅ Streamline Terraform and OpenTofu workflows for better infrastructure-as-code management✅ Leverage Databricks for efficient data streaming in Azure✅ Salesforce insights in BigQuery for unified analytics📚 Must-Read Books for Data & BI Professionals📖Causal Inference and Discovery in Python:Go beyond predictions with causal effect estimation in fraud, healthcare & more.📖The Definitive Guide to Power Query (M): Automate data prep, optimize workflows & streamline analytics.📖Bayesian Analysis with Python: Build Bayesian models with PyMC for smarter decisions, no stats needed!📖Mastering PyTorch: Learn CNNs, transformers, AutoML & cloud deployment.📖The Machine Learning Solutions Architect Handbook:Design & scale AI/ML like a pro.📖Mastering Tableau 2023:AI-powered visualizations & governance for BI analysts.🌟 BI & AI on the RiseThis week, we highlight AWS Pi Day 2025, Microsoft OneLake’s Iceberg integration, and Google’s Cloud Composer 3, all pushing the boundaries of data management, automation, and AI-driven insights. Plus, see how Definity Insurance transformed its analytics with BigQuery and Vertex AI, cutting migration time in half while unlocking real-time insights and AI-driven decision-making.⚡ Ready to dive in? Scroll down for the latest trends and expert insights!Cheers,Merlyn ShelleyGrowth Lead, Packt🎯 BI Mastery: The Ultimate Reading List for 2025💎 Causal Inference and Discovery in Python - By Aleksander MolakUnderstanding why something happens is key for data professionals. This hands-on Python guide covers causal effect estimation, discovery, and ML applications in fraud, healthcare, and more. Elevate your models beyond prediction, get your copy and master causal inference today!Buy eBook $31.99 $27.99💎 The Definitive Guide to Power Query (M) - By Greg Deckler, Rick de Groot, Melissa de KorteTired of manual data cleaning? Master Power Query to automate, optimize, and speed up workflows. This guide covers fundamentals, advanced M language, and performance optimization, helping analysts and BI pros streamline prep, save time, and enhance analytics. Get your copy today!Buy eBook $43.99💎 Bayesian Analysis with Python - By Osvaldo MartinGo beyond traditional stats with Bayesian analysis for confident, data-driven decisions. This Python guide covers modeling with PyMC, real-world applications, and model evaluation, ideal for data scientists, researchers, and developers. No prior stats experience needed, get your copy today!Buy eBook $39.99 $35.98💎 Mastering PyTorch - By Ashish Ranjan JhaMaster PyTorch for cutting-edge AI! This guide covers CNNs, transformers, diffusion models, multi-GPU training, AutoML, and deployment to mobile, cloud, and production. Ideal for data scientists, ML engineers, and researchers, get your copy and level up today!Buy eBook $41.99 $36.99💎 The Machine Learning Solutions Architect Handbook - By David PingDesign, deploy, and scale ML like an expert! Written by AWS’s David Ping, this guide covers ML lifecycle, enterprise AI architecture, and generative AI. Perfect for ML engineers, architects, and data scientists, get your copy and master ML solutions today!Buy eBook $39.99 $35.98💎 Mastering Tableau 2023 - By Marleen MeierMaster Tableau and transform raw data into insights! This guide covers data prep, visualization, AI integration, and governance. Perfect for analysts, BI pros, and data scientists, build impactful dashboards and optimize performance. Get your copy today!Buy eBook $39.99 $35.98📊 Data Viz Trends Shaping the Future of Insights⏩ How to Fully Automate Data Cleaning with Python in 5 Steps: As a Business Intelligence professional, you often deal with messy data. This blog helps you automate data cleaning using Python’s pandas library, covering missing values, standardization, outlier handling, and validation, so you can build a reliable, repeatable pipeline for accurate analysis.⏩ Top 5 Power BI Common Pitfalls: This blog highlights five common mistakes in Power BI projects and how to avoid them. It covers data modeling, ETL best practices, naming conventions, report performance, and source control, helping BI professionals build scalable, efficient, and well-structured Power BI solutions.⏩ Identify Anti-Patterns in SQL Server Queries: This blog explores how SQL Server 2022’s Query_AntiPattern Extended Event helps identify inefficient query patterns. It covers common anti-patterns like non-sargable queries, parameter sniffing, and implicit conversions, guiding you in optimizing queries for better performance and resource utilization.⏩ Digitally Signing a SQL Stored Procedure: This blog explains how to digitally sign SQL Server stored procedures using self-signed certificates. It covers creating certificates, adding signatures, verifying integrity, and detecting unauthorized modifications, helping database professionals ensure security and authenticity of SQL objects against accidental or malicious changes.📈 Dive into Databases: SQL Essentials⏩ Optimize Delta Tables with VACUUM in Microsoft Fabric: This blog explains how to optimize Delta tables in Microsoft Fabric using the VACUUM operation. It covers identifying stale files, automating cleanup, preventing storage bloating, and maintaining partitioned data efficiently, helping data engineers improve performance and reduce unnecessary storage costs.⏩ Python Modules for Developing Data Engineering Workloads: This blog explores essential Python modules for building data engineering pipelines, focusing on attrs, SQLAlchemy, and pandas. It covers their installation, use cases, examples, and caveats, helping data engineers develop scalable, efficient, and maintainable ETL/ELT workflows.⏩ Gauss-Seidel Method SQL Function to Solve Linear Equations: This blog demonstrates how to implement the Gauss-Seidel method in SQL Server to solve systems of linear equations. It explains the function logic, input format, and practical examples, helping database professionals apply iterative numerical solutions directly within SQL.⏩ Attribute-Level Governance Using Apache Iceberg Tables: This blog explains how to implement attribute-level governance using Apache Iceberg tables and AWS Lake Formation. It covers fine-grained access control, column and row-level security, and efficient data cataloging, helping organizations manage secure, scalable, and compliant data access across cloud environments.🔄 Real-World Transformation: How Gen BI Made Data Work⏩ Top Terraform and OpenTofu Tools to Use in 2025: Explore the top Terraform and OpenTofu tools for 2025, designed to enhance infrastructure management, security, and collaboration. This guide covers version control, automation, security scanning, cost estimation, and state management tools, helping DevOps teams optimize Infrastructure-as-Code workflows efficiently.⏩ Queries for Optimizing and Debugging PostgreSQL Replication: Learn how to monitor, optimize, and debug PostgreSQL replication with key SQL queries. This guide covers tracking replication lag, managing slots, cleaning up unused subscriptions, and improving logical replication performance, helping database administrators maintain efficient and reliable PostgreSQL replication setups.⏩ Data Streaming Databricks in Azure: This blog explores data streaming in Azure Databricks, comparing structured streaming and Auto Loader for ingesting files into Delta Lake. It covers implementation steps, best practices, performance considerations, and real-world examples to help data engineers build scalable streaming pipelines efficiently.⏩ Using SQL Server Stored Procedures with the Django ORM: This blog explores integrating SQL Server stored procedures with Django’s ORM. It covers calling procedures, handling parameters, managing transactions, capturing multiple result sets, and dealing with output parameters, all with step-by-step explanations and code snippets for practical implementation.⚡ Quick Wins: BI Hacks for Instant Impact⏩ Unlock the power of your Iceberg data in OneLake: This blog introduces Microsoft OneLake’s integration with Snowflake and Apache Iceberg tables, enabling seamless data sharing without duplication. It covers the latest updates, steps to get started, and upcoming features that enhance interoperability, performance, and schema-level data management in Fabric.⏩ AWS Data & AI Day Copenhagen showcases the latest innovations in analytics and machine learning: AWS Data & AI Day Copenhagen brought together industry leaders to showcase cutting-edge innovations in data analytics and AI. The event featured success stories from Basware, Novo Nordisk, and Casper’s Ice Cream, illustrating how businesses leverage Amazon QuickSight, SageMaker, and AWS AI services to drive transformation.⏩ Accelerate analytics and AI innovation with the next generation of Amazon SageMaker: Amazon SageMaker has evolved into a unified data and AI development environment, streamlining how organizations manage analytics, machine learning, and generative AI. With SageMaker Unified Studio, teams can access, analyze, and act on data seamlessly, integrating AWS services like Redshift, Athena, and Amazon Bedrock to accelerate innovation.⏩ Streamlined Multiomics Data Analysis Leveraging Illumina Software on AWS: Multiomics research is transforming biomedical science, but managing vast genomic, transcriptomic, and proteomic datasets presents challenges. Illumina’s AWS-powered informatics solutions, including DRAGEN, Illumina Connected Analytics, and Correlation Engine,help researchers analyze, integrate, and visualize complex multiomics data efficiently, unlocking new insights into disease mechanisms and biomarker discovery.🎤 Voices of BI: Lessons from Industry Experts⏩ AWS Pi Day 2025: Data foundation for analytics and AI: AWS Pi Day 2025 showcased the latest advancements in cloud data management, analytics, and AI, with a focus on Amazon S3 Tables, SageMaker Unified Studio, and SageMaker Lakehouse. These innovations streamline data access, accelerate AI development, and unify analytics workflows for seamless, scalable insights.⏩ Datastream extracts Salesforce Data cloud data: Google Cloud has expanded Datastream to support Salesforce Data Cloud, enabling seamless real-time data replication into BigQuery, Cloud Storage, and other destinations. This integration eliminates data silos, enhances analytics, and empowers businesses with unified insights across operational and SaaS data for better decision-making.⏩ Cloud Composer 3 for Apache Airflow: Google Cloud has announced Cloud Composer 3, the next-generation managed Apache Airflow service, designed to simplify data pipeline orchestration. With hidden infrastructure, enhanced performance, simplified networking, and per-task resource control, data teams can focus on workflows rather than maintenance, boosting efficiency, security, and scalability.⏩ Definity's leap to data agility with BigQuery and Vertex AI: Definity Insurance successfully modernized its data infrastructure by migrating to Google Cloud’s BigQuery and Vertex AI, replacing its legacy Cloudera platform in just 10 months. This transformation reduced costs, improved scalability, accelerated AI adoption, and enabled real-time analytics, enhancing customer experiences and operational efficiency.We’ve got more great things coming your way, see you soon!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

OpenAI’s Deep Research, Data Pruning MNIST, RAG pipeline with RedisVLLearn Smarter, Your Way!✨ Something big is brewing for Data Science, BI, and ML learners at Packt! Share your thoughts and grab a FREE AI Crash Course eBook! 🔥📚👉 Take the Survey Now!Let's make learning even more amazing, together! 💡Take the Survey Now!Hyperproof's 6th Annual IT Risk and Compliance Benchmark Report ReleasedGRC is no longer just a checkbox, it’s a competitive advantage.Hyperproof’s 6th Annual IT Risk & Compliance Benchmark Report reveals a major shift: organizations are maturing their GRC practices, centralizing teams, and increasing budgets. With 91% of companies now prioritizing compliance, the landscape is evolving fast.The key takeaway? Governance, risk, and compliance are now drivers of operational excellence and strategic growth. Hyperproof’s industry insights and new GRC Maturity Model equip organizations to stay ahead.📊 Get the full report & start building a stronger, more resilient GRC strategy today.Download the Report Now!Sponsored📬Welcome to BIPro #88 – Your Weekly Business Intelligence Boost! 🚀 Get ready to explore the latest breakthroughs in AI-powered analytics, cloud data solutions, and next-gen BI tools! This week, we’re diving into OpenAI’s Deep Research Agent, Microsoft Fabric Copilot for DAX, and Striim’s AI-driven mirroring for operational data. Plus, don’t miss our expert insights on data readiness, visualization enhancements, and seamless cloud migrations.Check out our top highlights and latest BI book releases to stay ahead in the data-driven world! Let’s dive in 👇📚 New Releases You Can't Miss:✦ Causal Inference in R✦ Python Feature Engineering Cookbook✦ Quantum Machine Learning and Optimisation in Finance🧮 This week’s highlights: ❯ MicroStrategy Offers Personalized Experiences with AI in Latest MicroStrategy ONE Release❯ Building your first RAG pipeline with RedisVL❯ Microsoft Fabric Copilot to write DAX queries in Power BI update❯ What OpenAI’s Deep Research Means for the Future of Data Science❯ Mirroring operational data for the AI era with Striim and Microsoft Fabric❯ Tips for migrating Oracle-based applications to Google Cloud❯ An Effective Approach for High Volume Data in Azure SynapseDive in and let this week’s insights supercharge your BI journey! 🚀Cheers,Merlyn ShelleyGrowth Lead, Packt📚 Packt Signature Series: New Releases You Can't Miss❯❯❯❯ Causal Inference in R: Written by Subhajit Das, this book offers a deep dive into causal inference using R, guiding readers through foundational concepts and advanced techniques like propensity score matching and instrumental variables.It helps you develop skills to construct and interpret causal models, address challenges in controlled experiments, and apply doubly robust estimation. With real-world case studies and hands-on examples, the book empowers readers to make informed, data-driven decisions by understanding and establishing causal relationships with precision.Buy eBook $35.99 $24.99❯❯❯❯ Python Feature Engineering Cookbook: Written by Soledad Galli, this third edition of the Python Feature Engineering Cookbook provides a complete guide to crafting powerful features for machine learning models. It covers practical solutions for common challenges, such as imputing missing values and encoding categorical variables, while optimizing data transformation processes.The book explores advanced techniques like feature extraction from dates, times, text, and time series data, as well as using tools like Featuretools and tsfresh. With step-by-step instructions and real-world examples, it helps readers build reproducible feature engineering pipelines, ultimately enhancing machine learning model performance.Buy eBook $35.99 $24.99❯❯❯❯ Quantum Machine Learning and Optimisation in Finance: Written by Antoine Jacquier and Oleksiy Kondratyev, this second edition of Quantum Machine Learning and Optimisation in Finance explores how quantum algorithms enhance financial modeling and decision-making. The book focuses on quantum machine learning (QML) and optimization algorithms, with an emphasis on near-term applications using NISQ systems.It offers practical insights into hybrid quantum-classical computational protocols and addresses the limitations of current quantum hardware. The authors provide an accessible yet rigorous approach to QML, covering topics like quantum neural networks, quantum annealing, and variational algorithms, equipping readers with the knowledge to apply quantum techniques in financial innovation.Buy eBook $35.99 $24.99📊 Data Viz Trends Shaping the Future of Insights❯❯❯❯ An Effective Approach for High Volume Data in Azure Synapse: Azure Synapse Analytics, an MPP database, enables efficient high-volume data loading using the COPY INTO command. Data ingestion leverages Parquet files for performance. Fact tables use hash-distributed dynamic partitioning for scalability. Monthly partitions optimize query performance, ensuring balanced data distribution and compression.❯❯❯❯ MicroStrategy Offers Personalized Experiences with AI in Latest MicroStrategy ONE Release: MicroStrategy ONE’s latest update focuses on enhancing AI-powered business intelligence by improving the Auto AI bot’s conversational abilities, personalization, and contextual understanding. It introduces new chart types, user feedback integration, and better AI deployment controls, making AI-driven analytics more intuitive and adaptable.❯❯❯❯ Using Blue/Green Deployment For (near) Zero-Downtime Primary Key Updates in RDS MySQL: This blog explains how Amazon RDS Blue/Green deployment enables modifying large tables using asynchronous replication, minimizing downtime. It covers creating a Green environment, altering table structures, restarting replication, and switching over. The process ensures a smooth transition while keeping the database synchronized and minimizing disruption to applications.❯❯❯❯ Building your first RAG pipeline with RedisVL: This blog details the journey of building a Retrieval Augmented Generation (RAG) pipeline using the Redis Vector Library. It covers setting up Redis, processing data with vector embeddings, designing a schema, performing semantic searches, and creating an AI assistant that retrieves context-aware insights from financial documents.❯❯❯❯ What is content-based filtering? This blog explores content-based filtering in recommender systems, explaining its machine learning techniques, advantages, and limitations. It compares content-based vs. collaborative filtering, highlighting their trade-offs. The blog also provides a Redis-powered tutorial on building a movie recommendation system using vector embeddings, semantic search, and metadata-driven filtering for personalized suggestions.📈 Dive into Databases: SQL Essentials❯❯❯❯ Deep Dive into WebSockets and Their Role in Client-Server Communication: This blog explores WebSockets and real-time communication, comparing them with polling, webhooks, and Server-Sent Events (SSE). It explains how WebSockets enable bidirectional, persistent connections ideal for chat apps, gaming, and live notifications. The blog details WebSocket handshakes, connection setup, efficiency benefits, and practical use cases for interactive, low-latency applications.❯❯❯❯ How to Share a Secret: Shamir’s Secret Sharing: This blog explains secret sharing and explores Shamir’s Secret Sharing, a cryptographic technique for securely distributing secrets among multiple parties. It covers how polynomial-based secret sharing works, its security properties, real-world applications (e.g., medical research, finance), advantages, limitations, and implementation details, ensuring data privacy while enabling controlled access.❯❯❯❯ Analyze Tornado Data with Python and GeoPandas: This blog explores tornado data analysis using NOAA’s public-domain database from 1950–2023. It details data retrieval, filtering, geospatial mapping with GeoPandas, and visualizing tornado occurrences. The project highlights regional tornado trends, the expansion of ‘Dixie Alley,’ and improvements in detection due to Doppler radar advancements, revealing shifting tornado patterns over time.❯❯❯❯ How to do Date calculations in DAX: This blog explores date calculations in DAX, focusing on the DATEADD() function for time-based analysis. It explains shifting dates by days, months, and years, handling weeks with alternative methods, and using TREATAS() and CALCULATETABLE() for dynamic filtering. Practical examples demonstrate how to apply these techniques in real-world data models.❯❯❯❯ How to Implement Guardrails for Your AI Agents with CrewAI: This blog explores implementing guardrails for AI agents using CrewAI, ensuring controlled, safe, and reliable outputs. It covers LLM safety concerns, CrewAI’s agent-task separation, workflow management with Flows, and real-time content verification. A practical example demonstrates multi-agent coordination, iterative text validation, and mitigating risks in AI-powered applications.🔄 Real-World Transformation: How Gen BI Made Data Work❯❯❯❯ Mirroring operational data for the AI era with Striim and Microsoft Fabric: This blog explores Striim’s partnership with Microsoft Fabric to enable real-time data integration and AI-driven analytics. It introduces SQL2Fabric-Mirroring, a low-latency, scalable solution for replicating on-premises SQL data to Microsoft Fabric OneLake, supporting AI, analytics, and decision-making. The blog highlights Change Data Capture (CDC), automated synchronization, and seamless cloud integration.❯❯❯❯ Microsoft Fabric January 2025 update: This blog highlights Microsoft Fabric’s latest updates, including NotebookUtils session management, enhanced COPY INTO permissions, Fabric REST APIs, and ALM improvements. It announces FabCon 2025, Power BI DataViz Championships, free DP-700 certification training, and Copilot AI enhancements. Key updates span Power BI, OneLake, Data Engineering, Data Warehouse, and Real-Time Intelligence innovations. ❯❯❯❯ Private Preview of Migration assistant for Fabric Data Warehouse: This blog introduces Microsoft Fabric’s Migration Assistant, designed to streamline SQL Server and Synapse migrations to Fabric Data Warehouse. Currently in Private Preview, it offers schema conversion, data migration, and AI-powered assistance. Organizations can join the preview, provide feedback, and collaborate with the product team before the public release.❯❯❯❯ Power BI January 2025 Feature Summary: The January 2025 Power BI update brings exciting new features to enhance data exploration and visualization. Users can now quickly analyze data with the “Explore this data” option and improved Treemap tiling methods. Updates include semantic model version history tracking, TMDL scripting (preview), and enhanced PowerPoint storytelling tools. AI-driven Copilot enhancements provide suggested questions for deeper insights. A new Snowflake connector and advanced visualizations like Lollipop Charts expand analytics capabilities. Additionally, Microsoft Fabric Conference 2025 registration is open, and the Fabric Data Engineer Certification (DP-700) is now available.❯❯❯❯ Microsoft Fabric Copilot to write DAX queries in Power BI update: Microsoft Fabric Copilot now enhances DAX query writing in Power BI with semantic model descriptions, synonyms, and sample values. This update improves query accuracy by leveraging metadata from tables, columns, and measures. Users can define descriptions for clarity, add synonyms for flexibility, and utilize sample values for context, streamlining data insights.⚡ Quick Wins: BI Hacks for Instant Impact❯❯❯❯ Gather organization-wide Amazon RDS orphan snapshot insights using AWS Step Functions and Amazon QuickSight: AWS customers can now automate orphaned RDS snapshot identification across accounts and regions using AWS Step Functions, Lambda, Glue, and QuickSight. This solution enhances visibility, optimizes cloud spend, and streamlines snapshot management with centralized insights. It leverages AWS Organizations, Athena, and S3, offering flexible deployment and automated monitoring via EventBridge.❯❯❯❯ The Apiphani Data Pipeline and AWS Services Industrialize Data Delivery for BI, ML, and AI: This blog explores how Apiphani, an AWS Partner, helps organizations industrialize data delivery and maximize the value of BI, ML, AI, and digital products through scalable, reusable data pipelines. It covers technology, operational models, and cultural transformation, demonstrating how businesses can accelerate data-driven decision-making, reduce costs, and improve governance. ❯❯❯❯ Hybrid big data analytics with Amazon EMR on AWS Outposts: This blog explores Amazon EMR on AWS Outposts, a hybrid big data analytics solution that brings the power of Amazon EMR to on-premises environments. It details how businesses can process petabyte-scale data while meeting data residency, compliance, and latency requirements. The blog also covers deployment architecture, data integration with Amazon S3, network optimization with AWS Direct Connect, and secure data access using AWS Glue and Lake Formation.❯❯❯❯ February 2025 Amazon QuickSight events: This blog highlights upcoming Amazon QuickSight events for February 2025, showcasing the latest advancements in BI and generative BI. Attendees can explore industry use cases, new features like Amazon Q, advanced visualizations, and prompted reports. The blog also provides details on virtual learning sessions, in-person meetups, and user groups, helping organizations stay updated on QuickSight innovations and best practices.🎤 Voices of BI: Lessons from Industry Experts❯❯❯❯ What OpenAI’s Deep Research Means for the Future of Data Science: This blog introduces OpenAI’s Deep Research Agent, a revolutionary tool that automates multi-step research, synthesizes diverse data sources, and delivers verified insights for data scientists. It highlights how Deep Research accelerates problem-solving in AI, healthcare, and finance, ensuring accuracy, efficiency, and scalability in tackling complex, domain-specific challenges with real-time, transparent data synthesis.❯❯❯❯ Tips for migrating Oracle-based applications to Google Cloud: This blog explores the Google Cloud-Oracle partnership, enabling businesses to migrate and modernize Oracle databases and applications on Google Cloud. It details migration paths, containerization with GKE and Cloud Run, Exadata integration, and Java optimization with GraalVM. Businesses benefit from scalability, security, and flexibility, accelerating cloud transformation, DevOps integration, and cost efficiency while leveraging Google’s high-performance infrastructure.❯❯❯❯ Open Mirroring for SAP sources – dab and Simplement: This blog highlights Fabric Mirroring, a data replication feature in Microsoft Fabric that ensures seamless synchronization of source data into Fabric OneLake. It introduces Open Mirroring, an extensible replication platform, now supporting SAP data integration. Partners like dab Nexus and Simplement Roundhouse enable efficient SAP data replication, enhancing data accessibility, analytics, and integration across Fabric workloads.❯❯❯❯ Data Pruning MNIST: How I Hit 99% Accuracy Using Half the Data. This blog explores data-centric AI and data pruning to improve model efficiency and accuracy. It demonstrates how the "furthest-from-centroid" selection strategy on MNIST achieves 98.73% accuracy using just 50% of the dataset. Key insights include reducing redundancy, enhancing decision boundaries, and optimizing dataset curation, challenging the assumption that more data always improves AI models.We’ve got more great things coming your way, see you soon!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Custom T-SQL in Azure Studio, Dataproc Serverless Gets a Boost, SCD vs Overwrite, Patient Jarvis @media only screen and (max-width: 100%;} #pad-desktop {display: none !important;} }Shouldn't GenAI be doing all the cyber crap jobs by now?Learn about the latest in GenAI for vulnerability management, exposure management and cyber-asset security when you attend the CyberRisk Summit.This free, virtual event on Wednesday, Nov. 20 includes expert speakers from Yahoo, Wells Fargo, IBM, Vulcan Cyber and more. This is the ninth, semi-annual CyberRisk Summit. Attendees can request CPE credits, and all registrants get access to the session recordings. Join us!Register for freeSponsored🗞️Welcome to BIPro #84 – Your Weekly Dose of BI Brilliance! 🚀Fuel your data-driven decisions with the freshest trends, strategies, and hacks from the world of business intelligence.📊 Data Viz & Tools: Future-Proof Your Insights◘ Pandas + SQL = Powerhouse Duo: Unleash their combined potential for seamless data analysis.◘ DuckDB Demystified: A Python-based guide to effortless analytics.◘ Google Cloud’s Secure Data Playbook: Step-by-step to building a fortress-like platform.◘ Custom T-SQL in Azure Studio: Speed up workflows with tailored code snippets.◘ Master Pandas for Data Wrangling: Learn the essentials to transform tabular data.◘ Small Deployments Made Easy: Cloud Migration App simplifies the process.◘ Alteryx Fall 2024 Updates: Faster workflows, better reports—dive in!🔄 BI in Action: Real-World Innovations◘ REST APIs & Fabric: Master the art of data ingestion.◘ GraphQL Meets Fabric: Discover powerful relationships through Microsoft’s API.◘ Dataproc Serverless Gets a Boost: Performance upgrades you can’t miss.◘ Index Management 101: Clean databases = fast queries.◘ Saving Big on Open-Source DBs: Proven cost-cutting strategies.◘ Sentiment Analysis with WebAssembly: SingleStore’s clever approach.◘ Topgolf’s BI Makeover: Learn how QuickSight transformed their game.⚡ Quick Wins: BI Hacks You’ll Love◘ Power BI Magic: Running totals, averages, and more with aggregate functions.◘ SQL Simplified: Clear examples of IS NULL and IS NOT NULL usage.◘ SCD vs Overwrite: Navigate data warehouse dimensions with ease.◘ Moving Averages Made Simple: T-SQL windowing functions explained.◘ Streaming Architecture 101: Build with Apache Kafka and Zookeeper.◘ Patient Jarvis Solution: Fractal’s innovative approach to patient insights.🎤 Voices of BI: Wisdom from the Experts◘ Tableau Viz Extensions: Everything you need to level up visualizations.◘ Graph It Right: NetworkX tips for mastering graphs in Python.◘ Data Validation Done Right: Introducing Pandera for Python users.◘ Fixing Cross-Validation Flaws: Common pitfalls and practical solutions.◘ 6 Pillars of Data Analysis: A framework for actionable insights.◘ AlloyDB Omni 15.7.0: What’s new and why it matters.Enjoy this week’s curated lineup of BI brilliance!Calling All Data & BI Enthusiasts!Do you dream of sharing your insights and building your reputation in the Data & BI community? Contribute to our new column in the Packt BIPro newsletter! Share your experiences, discuss new BI tools, or ask questions. Gain recognition among 37,000 BI professionals. Reply with your Google Docs article or use our weekly feedback form. Enjoy a free PDF of "Interactive Data Visualization with Python - Second Edition" for participating. Click reply or share your content today!Share your thoughts and opinions here!Cheers,Merlyn ShelleyEditor-in-Chief, PacktPackt’s Signature Series: New Titles Just Arrived!📚➽Learn Microsoft Fabric: Explore Microsoft Fabric's features through real-world examples to build robust data analytics solutions, including lakehouses and data warehouses. Learn to monitor and manage your analytics system for flexibility, performance, and security, while leveraging AI-driven insights with Copilot integration. Start your free trial for access, renewing at $19.99/month.eBook $24.99 $35.99Print + eBook $44.99➽Microsoft Power BI Cookbook - Third Edition: Dive into Microsoft Data Fabric to enhance data strategies and gain deeper insights. Effortlessly create Hybrid tables and comprehensive scorecards while utilizing new visualization tools that transform complex data into clear, actionable charts and reports for effective decision-making in Power BI. Start your free trial for access, renewing at $19.99/month.eBook $29.99 $43.99Print + eBook $59.99➽Fundamentals of Analytics Engineering: Explore how analytics engineering aligns with your organization's data strategy while gaining insights from seven industry experts. Address common challenges faced by businesses and learn to implement scalable analytics solutions, from data ingestion to visualization, using industry-leading tools. Start your free trial for access, renewing at $19.99/month.eBook $24.99 $35.99Print + eBook $44.99➽Getting Started with DuckDB: Utilize DuckDB to efficiently load, transform, and query diverse data sources and formats. Gain hands-on experience with SQL, Python, and R for data analysis, while exploring how open-source tools and cloud services enhance DuckDB’s versatile capabilities in the data ecosystem. Start your free trial for access, renewing at $19.99/month.eBook $29.99 $43.99Print + eBook $54.99📊 Data Viz Trends Shaping the Future of Insights⫸ Using Pandas and SQL Together for Data Analysis: This blog helps you understand when to use SQL and Python together for data manipulation, showcasing how PandaSQL bridges SQL's readability with Python's flexibility for seamless integration and analysis in data workflows.⫸ A Guide to Data Analysis in Python with DuckDB: This blog introduces DuckDB, a powerful in-process OLAP database that lets you seamlessly query pandas DataFrames, CSVs, and Parquet files using SQL in Python. Learn how to set it up, generate sample data, and perform data analysis effortlessly.⫸ Learn how to build a secure data platform with Google Cloud ebook: Discover how Google Cloud secures data-driven innovation in the Building a Secure Data Platform with Google Cloud ebook. Learn about advanced tools like encryption, access controls, and compliance monitoring to protect your data while enabling intelligent applications and fostering business growth.⫸ How to Develop Custom T-SQL Code Snippets in Azure Data Studio: This blog guides you on efficiently using and creating custom T-SQL code snippets in Azure Data Studio, helping streamline your workflows by automating repetitive tasks and enhancing productivity in your SQL development process.⫸ Explore Pandas in Python to Analyze and Manipulate Tabular Data: This blog introduces you to the Pandas library, showcasing its power in data analysis and manipulation in Python. Learn key features, installation steps, and practical use cases like creating Series, performing arithmetic operations, and applying aggregations.⫸ How to Use the Cloud Migration App for Small Deployments? This blog introduces the Cloud Migration App for Small Deployments, a tool designed for Tableau administrators to easily transition content, users, and workbooks from Tableau Server to Tableau Cloud. Learn its key features, setup process, and limitations for efficient small-scale migrations.⫸ Alteryx Fall 2024 Release Improves Workflow Efficiency and Reporting: This blog highlights the Fall 2024 Alteryx Release, offering simplified workflows, AI-powered reporting, and enhanced data connectivity. Discover new tools for cloud integration, hybrid architectures, and streamlined productivity to revolutionize data-driven decision-making for businesses and IT leaders.🔄 Real-World Transformation: How Gen BI Made Data Work⫸ Ingesting Data From REST API endpoints: Data Engineering with Fabric. This blog guides you through leveraging REST APIs in Python using a Spotify use case. Learn how to authenticate, retrieve data, handle errors, and interact with endpoints using dynamic functions—all within a Fabric notebook environment.⫸ Relationships with Microsoft Fabric GraphQL API: This blog explores using the Microsoft Fabric GraphQL API to query data across related tables in a star schema. Learn how to create relationships, handle directional queries, and implement advanced many-to-many relationships to maximize data accessibility for end-users.⫸ Dataproc Serverless performance and usability updates: This post introduces new features in Dataproc Serverless to enhance your Spark experience, including faster native query execution, real-time monitoring with a built-in Spark UI, and Gemini-powered autotuning for smarter troubleshooting and performance optimization.⫸ A Tidy Database is a Fast Database: Why Index Management Matters: This post is about identifying, optimizing, and managing database indexes to improve SQL Server performance. Learn how to address unused, fragmented, and overlapping indexes, resolve missing index issues, and implement effective maintenance strategies for efficient resource use and faster queries.⫸ Cost Optimization Strategies for Large-Scale Open-Source DBs: This post guides you on managing large-scale open-source databases cost-effectively. It covers choosing the right database, optimizing infrastructure, tuning performance, leveraging automation, and implementing strategies like caching, sharding, and containerization for efficiency and scalability.⫸ Using SingleStore and WebAssembly for Sentiment Analysis: This article guides you in performing sentiment analysis on Stack Overflow comments using SingleStore and WebAssembly, demonstrating data ingestion, function creation, and analysis through SQL and Python in the SingleStore Cloud environment.⫸ Transforming data into insights: How Topgolf revolutionized business intelligence using Amazon QuickSight. This post highlights how Topgolf transformed its operations with Amazon QuickSight, enabling organization-wide data access, real-time insights, and tailored dashboards to optimize performance, improve customer experiences, and foster a culture of data-driven decision-making.⚡ Quick Wins: BI Hacks for Instant Impact⫸ Aggregate Functions in Power BI - Running Total, Average, Max and Min: This post demonstrates how to create custom aggregations in Power BI using DAX (Data Analysis Expressions). Learn how to set up your data, build tailored measures, and gain precise insights to enhance your reports and data understanding.⫸ SQL IS NULL and SQL IS NOT NULL Examples: This post provides a clear guide on handling NULL values in SQL Server. Learn how to use IS NULL and IS NOT NULL operators effectively, understand the nuances of NULL, and avoid common pitfalls in SQL queries.⫸ Data Warehouse Considerations - SCD Type 2 vs Overwrite Dimensions: This post explores two key strategies for managing dimension table updates in data warehousing: Overwriting Tables and Slowly Changing Dimensions (SCD) Type 2. Learn their use cases, benefits, and why SCD Type 2 is often ideal for tracking historical data changes.⫸ Calculate a Moving Average with T-SQL Windowing Functions: This post explores two methods for calculating moving averages in SQL Server: an older self-join approach and a modern windowing function approach. Learn how to optimize queries and improve performance with indexes and efficient SQL techniques.⫸ Build a Streaming Data Architecture with Apache Kafka and Zookeeper: This article demonstrates how to use Apache Kafka and Zookeeper for real-time data streaming, showcasing a project to capture, process, and load data into Elasticsearch and Azure Data Lake Gen 2 for analysis.⫸ Revolutionizing Patient Insights with Fractal’s Patient Jarvis solution: This article introduces Fractal’s Patient Jarvis, an AI-powered solution designed to streamline pharmaceutical data analytics. It unifies claims data, leverages AWS-powered AI, and provides actionable insights to improve decision-making, operational efficiency, and patient outcomes in the pharmaceutical industry.🎤 Voices of BI: Lessons from Industry Experts⫸ Your Guide to Tableau Viz Extensions: This article highlights the revolutionary Viz Extensions in Tableau 2024.2, enabling the creation of complex visualizations—like Sankey diagrams, radar charts, and network diagrams—as easily as traditional charts, simplifying advanced analytics and expanding Tableau's capabilities.⫸ Navigating Networks with NetworkX: A Short Guide to Graphs in Python. This article introduces NetworkX, a Python library for building, analyzing, and visualizing networks, showcasing its applications in understanding complex relationships such as social connections or transportation systems through nodes and edges, enriched with attributes and algorithms.⫸ Data Validation with Pandera in Python: This article explores how Pandera, a Python library, streamlines data validation for dataframe-like objects in machine learning and analytics pipelines. It highlights Pandera's efficiency, scalability, and support for libraries like pandas and Dask, emphasizing its custom validations and schema-based approach to ensure data integrity.⫸ Why Most Cross-Validation Visualizations Are Wrong (And How to Fix Them)? This article critiques traditional cross-validation diagrams in data science, highlighting how they confuse the brain by making chunks of data appear as one moving piece. It proposes rethinking visuals to align with natural cognition and inclusivity.⫸ A Practical Framework for Data Analysis: 6 Essential Principles: This article outlines six essential data analysis principles for data scientists, focusing on techniques like establishing baselines, normalizing metrics, MECE grouping, aggregating data, removing irrelevant information, and applying the Pareto principle to extract actionable insights.⫸ What’s new in AlloyDB Omni version 15.7.0: The article highlights the new features in AlloyDB Omni version 15.7.0, including faster performance, an ultra-fast disk cache, an enhanced columnar engine,ScaNN vector indexing, and an updated Kubernetes operator, advancing PostgreSQL workflows across diverse environments.We’ve got more great things coming your way—see you soon!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}} @media only screen and (max-width: 100%;} #pad-desktop {display: none !important;} }

Threads in OpenAI Assistants API, SQL Dynamic Data Masking for Privacy and Compliance🌟Share, Shape, & Claim Your Free Packt Credit!📚 We're looking for data professionals to join a quick30-minute chatabout their learning needs. Thefirst 25 respondentsin a data-specific role will have the opportunity to speak with our team, share their insights, and receive afree Packt creditto claim any eBook of their choice! Hurry –submit your interest nowand keep an eye out for our team's meeting invite. You could be one of the chosen ones!👉 Reserve Your Interview SlotFortified’s Central Command Platform Named “Healthcare Cybersecurity Solution of the Year”Fortified Health Security’s Central Command platform has been named Healthcare Cybersecurity Solution of the Year by CyberSecurity Breakthrough. This unified platform streamlines risk tracking, threat monitoring, and real-time incident response, enhancing efficiency and patient protection. Learn more and see it in action today!Book a Demo Now!Sponsored🗞️Welcome to BIPro#90 – Your Weekly Business Intelligence Boost! 🚀Another week, another round of exciting updates in the world of data and BI! This time, we’re exploring SQL Database Project in Azure Data Studio, handling high-volume data in Azure Synapse, and unlocking the power of Key Vault services in Azure.We’ve also got some cool insights on Memorystore Cluster Autoscaler now on GitHub, Threads in OpenAI Assistants API, and how SQL Dynamic Data Masking helps with privacy and compliance. And if you're into Spring Data Neo4j, we've got something for you too!Plus, check out the latest BI book releases and top highlights to keep you ahead in this data-driven world. Let’s get into it! 👇📚 New Releases You Can't Miss:✦ Causal Inference in R✦ Python Feature Engineering Cookbook✦ Quantum Machine Learning and Optimisation in FinanceDive in and let this week’s insights supercharge your BI journey! 🚀Cheers,Merlyn ShelleyGrowth Lead, PacktDetect shadow AI hidden in the apps you build or useCISOs face growing pressure to govern AI usage in their organizations, but shadow AI is creeping into mobile apps, often unnoticed. With third-party SDKs making up 60-70% of app code, security risks are everywhere. NowSecure helps security teams detect undeclared AI in mobile apps, ensuring compliance and protecting sensitive data. Book a demo today to take control of your AI governance! 👉Book a Call to Assess Your AI RisksSponsored📚 Packt Signature Series: New Releases You Can't Miss❯❯❯❯ Causal Inference in R: Written by Subhajit Das, this book offers a deep dive into causal inference using R, guiding readers through foundational concepts and advanced techniques like propensity score matching and instrumental variables.It helps you develop skills to construct and interpret causal models, address challenges in controlled experiments, and apply doubly robust estimation. With real-world case studies and hands-on examples, the book empowers readers to make informed, data-driven decisions by understanding and establishing causal relationships with precision.Buy eBook $35.99 $24.99❯❯❯❯ Python Feature Engineering Cookbook: Written by Soledad Galli, this third edition of the Python Feature Engineering Cookbook provides a complete guide to crafting powerful features for machine learning models. It covers practical solutions for common challenges, such as imputing missing values and encoding categorical variables, while optimizing data transformation processes.The book explores advanced techniques like feature extraction from dates, times, text, and time series data, as well as using tools like Featuretools and tsfresh. With step-by-step instructions and real-world examples, it helps readers build reproducible feature engineering pipelines, ultimately enhancing machine learning model performance.Buy eBook $35.99 $24.99❯❯❯❯ Quantum Machine Learning and Optimisation in Finance: Written by Antoine Jacquier and Oleksiy Kondratyev, this second edition of Quantum Machine Learning and Optimisation in Finance explores how quantum algorithms enhance financial modeling and decision-making. The book focuses on quantum machine learning (QML) and optimization algorithms, with an emphasis on near-term applications using NISQ systems.It offers practical insights into hybrid quantum-classical computational protocols and addresses the limitations of current quantum hardware. The authors provide an accessible yet rigorous approach to QML, covering topics like quantum neural networks, quantum annealing, and variational algorithms, equipping readers with the knowledge to apply quantum techniques in financial innovation.Buy eBook $35.99 $24.99📊 Data Viz Trends Shaping the Future of Insights❯❯❯❯ SQL Database Project in Azure Data Studio: This article explains how to use the Azure Data Studio extension for managing SQL Database projects. It covers installation, project creation from existing databases or from scratch, adding tables, creating views, and stored procedures. The guide also emphasizes version control in Visual Studio and simplifies publishing changes.❯❯❯❯ An Effective Approach for High Volume Data in Azure Synapse: This article outlines an efficient approach for handling high-volume data in Azure Synapse Analytics. It covers parallel data loading using the COPY INTO command, leveraging Parquet files for efficiency, and implementing dynamic partitioning in fact tables. The method ensures optimal query performance by maintaining balanced distributions and sufficient row counts per partition.❯❯❯❯ JSON in Microsoft SQL Server: A Comprehensive Guide: This article explores handling JSON data in Microsoft SQL Server, covering storage, retrieval, validation, querying, modification, and performance optimization. It demonstrates using built-in functions like JSON_VALUE, JSON_QUERY, OPENJSON, and JSON_MODIFY, while ensuring data integrity with ISJSON() constraints. Best practices include indexing computed columns, schema validation with stored procedures, and error handling to maintain efficient and secure JSON operations in SQL Server.❯❯❯❯ Creating a Linked Server in Amazon RDS for SQL Server: A Step-by-Step Guide. This guide explains how to create and configure a linked server in Amazon RDS for SQL Server using SQL commands. It covers prerequisites, authentication setup, testing, and advanced configurations like timeout settings and remote procedure calls. Best practices include using linked servers sparingly, securing connections, and optimizing queries for performance.📈 Dive into Databases: SQL Essentials❯❯❯❯ Using Key Vault services in Azure Ecosystem: This guide explains how to use Azure Key Vault to securely store and manage secrets like passwords and access keys. It covers creating a Key Vault, storing secrets, and setting up access permissions using Access Control (IAM) and Access Policies. Applications can retrieve secrets securely, reducing the need to store sensitive information in code.❯❯❯❯ Software Deployment Strategies: This article explores software deployment strategies, focusing on Canary and Blue-Green deployments. Canary deployment gradually releases updates to a small group of users, ensuring stability before a full rollout. Blue-Green deployment runs two environments in parallel, enabling instant rollback if needed. Both strategies minimize downtime and risks, with trade-offs in complexity and cost.❯❯❯❯ Support Vector Machines: A Progression of Algorithms. This article explains the progression of Support Vector Machines (SVMs) from Maximal Margin Classifier (MMC) to Support Vector Classifier (SVC) and finally to full SVM. MMC finds a strict linear boundary, SVC allows some misclassification, and SVM extends this by using kernel functions to classify non-linear data efficiently.❯❯❯❯ Accelerate migration from traditional BI tools to Amazon QuickSight with generative AI and Storm Reply. This article details BMW Group's migration from on-premises BI tools to Amazon QuickSight, leveraging automation and generative AI. The project streamlined dashboard conversions, reducing manual effort by 80% while maintaining 90% data accuracy. The approach improved scalability, simplified BI processes, and demonstrated the potential of AI-driven cloud BI modernization.🔄 Real-World Transformation: How Gen BI Made Data Work❯❯❯❯ Deep Dive into WebSockets and Their Role in Client-Server Communication. This blog thoroughly examines real-time communication methods, focusing on WebSockets and their role in enabling two-way interactions. It explains how WebSockets differ from traditional HTTP approaches, outlines design challenges for messaging apps, and discusses scaling strategies, reliability, and best practices.❯❯❯❯ Amazon Redshift Serverless adds higher base capacity of up to 1024 RPUs. This blog explains how Amazon Redshift Serverless transforms data warehousing by scaling compute resources with a new 1024 RPU capacity. It compares performance against 512 RPUs for complex queries, data ingestion, and analytics, emphasizing cost efficiency and faster execution times.❯❯❯❯ Governing the ML lifecycle at scale, Part 4: Scaling MLOps with security and governance controls. This blog outlines building and governing a multi-account machine learning platform for streamlined model deployment. It describes roles, standardized templates, secure provisioning, and automation that empower data science teams to transition models into production efficiently while ensuring governance and collaboration.❯❯❯❯ Handle errors in Apache Flink applications on AWS. This blog explains error handling in streaming applications using Apache Flink. It details proven strategies for managing errors through retries and dead letter queues. The post shows how asynchronous I/O and side outputs effectively preserve data integrity and boost reliability.⚡ Quick Wins: BI Hacks for Instant Impact❯❯❯❯ Memorystore Cluster Autoscaler now on GitHub. This article is about the open-source Memorystore Cluster Autoscaler for Redis on Google Cloud. It explains how the tool automatically scales Redis clusters, adjusting shard count based on CPU and memory usage, to optimize performance and manage costs. The article details its architecture, deployment options via Cloud Run or GKE, and various configuration scenarios for different workload patterns.❯❯❯❯ New query insights capabilities for Cloud SQL Enterprise Plus. This article introduces the new query insights enhancements for Cloud SQL Enterprise Plus edition. It explains how detailed telemetry, 30-day query plans, wait event analysis, index recommendations, and an AI-powered chat interface empower developers and DBAs to quickly diagnose and optimize high-performance databases on Google Cloud.❯❯❯❯ Spectra Logic Offers 24G Optical SAS Switch to Transform Data Center Tape Storage. This blog introduces Spectra Logic's OSW-2400 Optical SAS Switch, a new solution that transforms tape storage connectivity in data centers. It explains how active optical cables extend connection distances up to 100 meters, enabling flexible deployments, improved performance, and significant cost savings by reducing the need for expensive Fibre Channel infrastructure.❯❯❯❯ A Guide to Using Amazon Bedrock Prompts for LLM Integration: This blog introduces Amazon Bedrock, a fully managed service that simplifies integrating large language models into applications. It outlines key benefits like access to diverse models, enhanced security, and serverless operation, while providing hands-on Python examples, prompt management strategies, and best practices for production usage.🎤 Voices of BI: Lessons from Industry Experts❯❯❯❯ An In-Depth Guide to Threads in OpenAI Assistants API: This blog compares the limitations of standard chat completion models with the enhanced capabilities of the Assistance API. It explains how the Assistance API overcomes issues like lack of memory, computational limitations, and synchronous processing by supporting features such as persistent threads, code interpretation, file retrieval, function calling, and asynchronous workflows. The post includes Python code examples demonstrating how to create, list, retrieve, modify, and delete threads and messages, helping developers manage conversation context more effectively.❯❯❯❯ Indexed View for Aggregating Metrics: This blog explores using Microsoft Azure SQL for storing and querying daily user metrics in web applications. It demonstrates how to aggregate data, such as user activity from a hotel booking site, over daily, weekly, or monthly intervals, and highlights the performance benefits of using indexed views for real-time analytics on large datasets.❯❯❯❯ Spring Data Neo4j: How to Update an Entity: This blog explores various methods for updating entities in Spring Data Neo4j. It highlights the limitations of the default save () method, which can inadvertently overwrite existing values with null, and demonstrates alternative approaches such as PATCH methods, custom Cypher queries, and DTO-based projections to update only specific properties while preserving existing data.❯❯❯❯ SQL Dynamic Data Masking for Privacy and Compliance: This blog explains SQL Server Dynamic Data Masking, a feature that obscures sensitive data from non-privileged users to enhance security and compliance. It covers when and why to use masking (e.g., in development environments, for third-party access, and to meet regulatory requirements), outlines prerequisites and masking functions, and provides step-by-step examples for applying and testing masking rules. The post also discusses how dynamic masking supports data minimization, audit readiness, and scalability, ensuring only authorized users see full data while others view masked values.We’ve got more great things coming your way, see you soon!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

The AI Value Playbook, The Definitive Guide to Power Query (M), Microsoft Power BI Cookbook @media only screen and (max-width: 100%;} #pad-desktop {display: none !important;} }3 Days. 25+ AI Experts. 30+ Sessions.Join the Generative AI In Action conference from Nov 11-13 (LIVE | Virtual) and gain insights from top AI leaders across over 30 sessions. Explore key topics including GenAI tools, AI Agents, Open-Source LLMs, Small Language Models, LLM fine-tuning, and many more! This is your opportunity to dive deep into cutting-edge AI strategies and technologies.Save 40% with our Early Bird offer using code BIGSAVE40 – don’t miss out!Secure Your Seat Today!🦋 Welcome to BIPro #76 – Your Weekly Business Intelligence Power-Up! 🚀Gear up for a fresh batch of BI trends, cutting-edge strategies, and top insights to supercharge your data journey!📚 Must-Read BI Books of the Week✦ The AI Value Playbook: How to Make AI Work in the Real World✦Building LLM-Powered Apps: Level up with AI-driven applications.✦Python for Algorithmic Trading Cookbook: Unleash the power of Python in trading.✦Microsoft Power BI Cookbook (3rd Ed.): Master Power BI like a pro.✦The Definitive Guide to Power Query (M): Dominate data wrangling in Power BI.✦Mastering PyTorch (2nd Ed.): Deep dive into PyTorch for AI innovation.🎯 Handpicked Articles Just for You!✦Master Data Transformation with R & dplyr: Elevate your data manipulation game.✦Eigenvalues & Eigenvectors in NumPy: Tackle advanced math with Python.✦Import Data into BigQuery – Here’s How: Seamlessly load data like a pro.✦Visualize with ggplot2 in R: Transform raw data into stunning visuals.✦Query Microsoft Fabric GraphQL API: Easily integrate with external apps.Stay ahead of the curve with these latest insights!Calling All Data & BI Enthusiasts!Do you dream of sharing your insights and building your reputation in the Data & BI community? Contribute to our new column in the Packt BIPro newsletter! Share your experiences, discuss new BI tools, or ask questions. Gain recognition among 37,000 BI professionals. Reply with your Google Docs article or use our weekly feedback form. Enjoy a free PDF of "Interactive Data Visualization with Python - Second Edition" for participating. Click reply or share your content today!Share your thoughts and opinions here!Cheers,Merlyn ShelleyEditor-in-Chief, PacktPackt’s Signature Series: New Titles Just Arrived!📚➽ Building LLM Powered Applications: This new titleis all about helping engineers and data pros use large language models (LLMs) effectively. It tackles key challenges like embedding LLMs into real-world apps and mastering prompt engineering techniques. You’ll learn to orchestrate LLMs with LangChain and explore various models, making it easier to create intelligent systems that can handle both structured and unstructured data. It’s a great way to boost your skills, whether you’re new to AI or already experienced! Start your free trial for access, renewing at $19.99/month.eBook $27.98 $39.99Print + eBook $34.98 $49.99➽ Microsoft Power BI Cookbook - Third Edition: The Power BI Cookbook is your essential guide to mastering data analysis and visualization with Power BI. It covers using Microsoft Data Fabric, managing Hybrid tables, and creating effective scorecards. Learn to transform complex data into clear visuals, implement robust models, and enhance reports with real-time data. This updated edition prepares you for future AI innovations, making it a must-have for beginners and seasoned users alike! Start your free trial for access, renewing at $19.99/month.eBook $29.99 $43.99Print + eBook $41.98 $59.99➽ The Definitive Guide to Power Query (M): The Definitive Guide to Power Query (M) focuses on mastering data transformation with Power Query. It covers fundamental and advanced concepts through hands-on examples that address real-world problems. You'll learn the Power Query M language, optimize performance, handle errors, and implement efficient data processes. By the end, you'll have the skills to enhance your data analysis effectively! Start your free trial for access, renewing at $19.99/month.eBook $43.99Print + eBook $37.99 $54.99💡 Expert Insights from the Packt Community 🚀Introducing The AI Value Playbook: How to Make AI Work in the Real WorldBy Lisa Weaver-Lambert, Data and AI Leader in Capital Markets, formerly Microsoft, and AccentureAre you a business leader or board member intrigued by the groundbreaking advances in Generative AI (GenAI) and Large Language Models (LLMs)?If you want to quickly formulate a perspective on how to integrate AI, The AI Value Playbook by Lisa Weaver-Lambert, is a must read. This book addresses the gap in data and AI knowledge in leadership teams that have an appetite for nuanced, targeted and practical solutions. It includes which levers and processes to consider to future-proof businesses. The AI Value Playbook draws on conversations and case studies with leading practitioners across sectors and geographies who share their first-hand experiences successfully driving AI value and pathways for progress.Why is This Book a Must-Read for Business Leaders?Business leaders are challenged by the speed of AI innovation and how to navigate disruption and uncertainty. This book is a crucial resource for those who want to understand how to leverage AI to drive business value, drawn from the firsthand experience of those who have been implementing this technology successfully. In a series of over 30 in-depth and wide-ranging conversations with practitioners, from CEOs leading new generative AI-based companies to Data Scientists and CFOs working in more traditional companies share their hard-earned wisdom. They talk candidly about their successes and failures, and what excites them about the future. These interviews offer unique insights for business leaders to apply to their own organizations. The book distils a value-driven playbook for how AI can be put to work today.Experts include:Sam Liang, CEO of Otter.aiAmr Awadallah, Founder and CEO at VectaraPhilipp Heltewig, Co-Founder and CEO at CognigyJoshua Rubin, Principle AI Scientist at Fiddler AIZeev Farbman, Co-Founder & CEO at Lightricks…and many more innovators who are actively shaping the AI landscape.Key Topics Covered in the PlaybookThis book provides case studies which explore the specifics of real-world applications. These present detailed analyses of practical scenarios, offering a closer look at the application and impact of AI, such as:How Generative AI Transforms Healthcare Education (LLMs & RAG enabling hyper-personalized learning for healthcare technicians)AI-Powered Virtual Agents Improving Service Efficiency (Real-world examples of AI's impact on customer service operations)Unlocking Profit with AI (Leveraging enterprise data for increased customer profitability and minimizing churn)The Role of Multimodal LLMs in Software Development (Innovations that redefine customer interaction and product creation)The last section of the book is The ‘AI Value Playbook’ a practical framework distilled from the experts and Lisa’s own professional experience, for successful AI implementation. Answers to the Big Questions for Business LeadersThe book tackles the pressing questions business leaders are facing today, such as:How can organizations adapt to the rapid pace of AI innovation?How do we strategically deploy AI to enhance efficiency and drive business value?What risks and ethical considerations should be addressed?How quickly can we start seeing measurable benefits from AI integration?What You’ll Take AwayThe AI Value Playbook distils a value-driven playbook for how AI can be put to work today, including:Fundamentals of AI concepts and the tech stackHow AI works with real-world practical applicationsHow to integrate into your company’s overall strategyHow to incorporate generative AI in your processesHow to drive value with sector-wide examplesHow to organize an AI-driven operating modelHow to use AI for competitive advantageThe dos and don’ts of AI applicationWith endorsements from Said Business School, University of Oxford, Microsoft leaders, Private Equity and Venture Capital leaders and board leaders, don't miss out on this opportunity to learn from the practical scenarios and strategic plays. The AI Value Playbook is a versatile resource and roadmap to making AI work in the real world—starting today.Get Your Copy Today and Start Driving Real AI Value!📊 Data Viz Trends Shaping the Future of Insights➽ How to Use R for Data Transformation with dplyr? This blog explores overcoming challenges in fine-tuning and deploying large language models (LLMs) using R's 'dplyr' package. It covers installation, selecting and renaming columns, and filtering rows to streamline data transformation for effective analysis in R.➽ How to Calculate Eigenvalues and Eigenvectors with NumPy? This blog explains how to calculate eigenvalues and eigenvectors using NumPy's linear algebra module. It covers the mathematical background, provides practical coding examples, and discusses the implications of eigenvalues in applications like Principal Component Analysis (PCA) for dimensionality reduction.➽ How to Import Data into BigQuery? This tutorial demonstrates how to load datasets into Google BigQuery from various sources, including CSV, JSON, Google Cloud Storage, and Google Sheets. It outlines prerequisites, interface navigation, and step-by-step procedures for each data loading method, focusing on Asian cuisine examples.➽ How to Visualize Data with ggplot2 in R? This article introduces ggplot2, an R package for creating visualizations like scatter plots, line plots, bar plots, and more. It covers installation, basic and advanced plot types, and how to save plots, helping users effectively visualize data.🔄 Real-World Transformation: How Gen BI Made Data Work➽ How to Evaluate RAG If You Don’t Have Ground Truth Data? This article discusses strategies for evaluating Retrieval-Augmented Generation (RAG) models without ground truth data. It covers retrieval and generation evaluations, including using vector similarity thresholds, multiple LLMs for response comparison, and human feedback to establish criteria for relevance, correctness, and fluency. Additionally, it outlines methods to create a ground truth dataset from scratch using existing datasets or manual curation.➽ Start Asking Data Why? This blog explores how to uncover causal relationships in observational data without relying on expensive randomized control trials. It emphasizes the importance of understanding the story behind the data, introduces causal reasoning through Simpson’s and Berkson’s Paradoxes, and advocates for using causal graphs to enhance data analysis.➽ Choosing Between LLM Agent Frameworks: This blog discusses the evolving landscape of AI agents, highlighting the shift from Retrieval-Augmented Generation (RAG) to modern frameworks for developing autonomous systems. It reviews various agent frameworks, compares their strengths and weaknesses, and provides insights on building agents from scratch, including challenges and benefits.➽ Your Documents Are Trying to Tell You What’s Relevant: Better RAG Using Links. This article addresses the challenges in building retrieval-augmented generation (RAG) applications, particularly in document retrieval. It introduces a new data model—linked documents—that enhances performance by preserving references like citations and hyperlinks, improving the ability to find relevant information. The author discusses the limitations of vector searches and emphasizes the importance of document structure and connections. By implementing document linking, the article illustrates how to effectively retrieve and utilize related documents, ultimately enriching the response quality of RAG systems.⚡ Quick Wins: BI Hacks for Instant Impact➽ Query Microsoft Fabric GraphQL API from an External App: This guide outlines how to query data from a Microsoft Fabric workspace using a C# application via the Microsoft Fabric GraphQL API. It details prerequisites, including creating a Microsoft Entra app and configuring necessary permissions for data access.➽ Building an Interactive UI for Llamaindex Workflows: This article expands on using LlamaIndex workflows to enhance research and presentations by integrating a Streamlit UI. It outlines how to create a user-friendly interface that displays progress, collects user input, and generates downloadable slide decks while detailing backend enhancements and streaming event implementations.➽ Achieve near real-time analytics on Amazon DynamoDB with SingleStore: This article outlines methods for integrating Amazon DynamoDB with SingleStore for near real-time analytics. It describes two architectural patterns: using DynamoDB Streams with AWS Lambda and leveraging Amazon Kinesis Data Streams with Amazon MSK, enabling efficient data capture and analysis.➽ How AI Platforms Are Transforming Business Data Management? The article discusses the rapid growth of the AI market, valued at $196.63 billion in 2023, projected to grow at a CAGR of 36.6% through 2030. It highlights challenges in traditional data management, with the global data management market valued at $89.34 billion in 2022. The rise of platforms like Databricks and Snowflake is emphasized, with Databricks valued at $43 billion in 2023, contributing to an anticipated data analytics market value of about $550 billion by 2028. The piece emphasizes the importance of data quality and integration of open-source and closed-source models, advocating for industry-specific AI solutions for better performance.🎤 Voices of BI: Lessons from Industry Experts➽ Introducing the OpenAI Academy: OpenAI is launching the OpenAI Academy to invest in developers and organizations in low- and middle-income countries, providing training, technical guidance, and $1 million in API credits. The initiative aims to enhance local AI talent, drive economic growth, and foster community innovation.➽ Looker Chart Config Editor tips: Google Cloud is enhancing Looker’s capabilities with new visualization options, including bullet charts and sunbursts, accessible through the Chart Config Editor. This post shares tips on using the Highcharts API to improve data visualizations, emphasizing customization and scrolling features for better user experience.➽ BigQuery jobs explorer is now GA: Google Cloud has launched BigQuery Jobs Explorer, a tool that provides comprehensive visibility into SQL query activity within organizations. It enables real-time monitoring, troubleshooting, and performance optimization, allowing users to track resource usage, identify costly queries, and improve overall efficiency.➽ Create security observability using generative AI with Security Lake and Amazon Q in QuickSight: This blog discusses a serverless solution for querying Amazon Security Lake data using natural language through Amazon QuickSight’s Amazon Q. It highlights the benefits of integrating generative AI for security use cases, improving threat response and data analysis by leveraging CloudTrail logs, VPC Flow Logs, and AWS services.We’ve got more great things coming your way—see you soon!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{line-height:0;font-size:75%} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}} @media only screen and (max-width: 100%;} #pad-desktop {display: none !important;} }

For a limited time, save on the best-selling books that will elevate your skills and knowledge! @media only screen and (max-width: 100%;} #pad-desktop {display: none !important;} }Introducing A Market-Changing Approach to Mobile App Protection by GuardsquareMobile applications face constant, evolving threats; to address these challenges, Guardsquare is proud to announce the launch of our innovative guided configuration approach to mobile app protection. By combining the highest level of protection with unparalleled ease of use, we empower developers and security professionals to secure their applications against even the most sophisticated threats. Guardsquare is setting a new standard for mobile app protection and we invite you to join us on this journey to experience the peace of mind that comes with knowing your mobile applications are protected by the most advanced and user-friendly product on the market.LEARN MORESponsored✨ Welcome to Packt’s Signature Series: New Titles Just Arrived!📚 We're thrilled to introduce the latest addition to our Signature Series—a curated collection of the best-selling titles in the data industry! This limited-time offer is packed with expert insights on mastering data science algorithms, Generative AI, and multimodal systems.For a limited time, enjoy 50% off eBooks and 30% off print editions of the following must-read titles. But hurry—this offer is only valid until September 30th!Don't miss this opportunity to upskill and elevate your career. Ready to dive in?Shape the Future of Development and Win Big!Join the Developer Nation Survey! Share how coding has evolved in 2024 and help steer tech innovation. Complete the quick survey for a chance to win amazing prizes like a Samsung Galaxy Watch, Raspberry Pi 5, and more! Plus, your participation supports worthy causes. Don’t miss out!TAKE THE SURVEYSponsored➽ AI-Assisted Programming for Web and Machine Learning: Unlock the power of AI-assisted programming to streamline web development and machine learning. Learn to enhance frontend and backend coding, optimize ML models, and automate tasks using GitHub Copilot and ChatGPT. Perfect for boosting productivity and refining workflows. Start your free trial for access, renewing at $19.99/month.eBook $18.99 $38.99Print + eBook $32.99 $47.99➽ Machine Learning and Generative AI for Marketing: Leverage AI and Python to revolutionize your marketing strategies with predictive analytics and personalized content creation. Learn to combine advanced segmentation techniques and generative AI to boost customer engagement while ensuring ethical AI practices. Perfect for driving real business growth. Start your free trial for access, renewing at $19.99/month.eBook $19.99 $39.99Print + eBook $34.98 $49.99➽ Amazon DynamoDB - The Definitive Guide: Master Amazon DynamoDB with this comprehensive guide, learning key-value data modeling, optimized strategies for transitioning from RDBMS, and efficient read consistency. Discover advanced techniques like caching and analytics integration with AWS services to boost performance, while minimizing latency and costs. Start your free trial for access, renewing at $19.99/month.eBook $17.99 $35.99Print + eBook $30.99 $44.99➽ Microsoft Power BI Performance Best Practices - Second Edition: Master Power BI performance optimization with this guide, learning to build efficient data models, apply row-level security, and troubleshoot issues using DAX Studio and VertiPaq Analyzer. Implement formal performance management strategies to ensure scalable, high-performing solutions. Start your free trial for access, renewing at $19.99/month.eBook $19.99 $39.99Print + eBook $34.98 $49.99➽ Polars Cookbook: Unlock faster, more efficient data analysis with Python Polars through step-by-step recipes. Master data manipulation, advanced querying, and performance optimization. Learn to handle large datasets, perform complex transformations, and integrate Polars with other tools. Start your free trial for access, renewing at $19.99/month.eBook $17.99 $35.99Print + eBook $30.99 $44.99➽ 15 Math Concepts Every Data Scientist Should Know: Master key data science algorithms through Python-based examples, boosting your solutions by applying and creating algorithms. Learn foundational and advanced mathematical techniques for solving real-world data challenges, with practical Python applications. Start your free trial for access, renewing at $19.99/month.eBook $17.99 $35.99Print + eBook $30.99 $44.99➽ Generative AI-Powered Assistant for Developers: Unlock the full potential of Amazon Q Developer with this comprehensive guide. Learn to auto-generate code across multiple languages, enhance productivity, and streamline workflows with generative AI. Includes real-world examples with AWS integration tips. Start your free trial for access, renewing at $19.99/month.eBook $15.99 $31.99Print + eBook $27.98 $39.99➽ Python Feature Engineering Cookbook - Third Edition: Streamline your machine learning workflows with this comprehensive guide to feature engineering. Learn to craft powerful features from tabular, transactional, and time-series data, develop reproducible pipelines, and optimize transformations to save time. Includes real-world examples for practical application. Start your free trial for access, renewing at $19.99/month.eBook $17.99 $35.99Print + eBook $30.99 $44.99Eager for more insights? Add these powerful resources to your reading list.➽ Bayesian Analysis with Python - Third Edition: Gain hands-on expertise in Bayesian modeling with PyMC, Bambi, and ArviZ. Explore hierarchical models, regression, and BART while applying best practices through practical exercises. Perfect for mastering real-world data science challenges. Includes a free PDF with book purchase.➽ Multiphysics Modeling Using COMSOL 5 and MATLAB: Master COMSOL and MATLAB integration with this comprehensive guide. Learn to set up and solve multiphysics models, from 0D to 3D, through practical examples. Advanced techniques like bioheat and Perfectly Matched Layer models are included, enhancing real-world engineering applications.➽ Python 3 Data Visualization Using ChatGPT / GPT-4: Master Python programming and data visualization with this comprehensive guide. Learn fundamentals and advanced techniques using libraries like Matplotlib and Seaborn. Explore AI integration with ChatGPT/GPT-4 for dynamic visualizations. Companion files with code, datasets, and figures enhance your hands-on learning experience, making this an essential resource for data scientists and Python practitioners.➽ Dealing With Data Pocket Primer: This complete guide covers data science fundamentals, from probability and statistics to advanced NLP and data visualization. Featuring practical examples, clear explanations, and companion files with source code, it’s the perfect resource for mastering data management and analysis efficiently.Here are some more fresh reads, handpicked just for you: ⏩ SQL Pocket Primer⏩ Data Visualization for Business Decisions⏩ Google Gemini for Python⏩ Enterprise Transformation to Artificial Intelligence and the Metaverse⏩ Pandas Basics⏩ Python 3 and Data Visualization⏩ Python 3 Data Visualization Using Google Gemini⏩ Python 3 Using ChatGPT / GPT-4We’ve got more great things coming your way—see you soon!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{line-height:0;font-size:75%} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}} @media only screen and (max-width: 100%;} #pad-desktop {display: none !important;} }