Posts

Handling Missing Data - Part 1

-
One of the big topics I have been speaking about is “ Data Cleansing for Machine Learning ”.  A major part of the data cleansing process is filling in missing values.  These values could be missing for a variety of reasons (manually entered data, improper data collection, incorrect exception handling, etc.)  Missing data can be summed up into 3 categories: MAR: Missing at Random, for patterns tied to known variables (e.g., age, education) where someone did not want to answer a question. MCAR: Missing Completely at Random, for random glitches, accidental loss (i.e. sensor failure) MNAR: Missing Not at Random, describes a situation where data is missing because of the value of the missing data itself.   To counter the issue of missing data, several techniques are available to fill in missing data.  In my video , I discuss 3 of those techniques. Multiple Imputation by Chained Equations (MICE) handles numeric or...
Post a Comment
Read more

Mar '26 Regional Tech Events

-
User Groups Mar 3: Ohio North Database Training Mar 11: Azure Cleveland Mar 19: GLUG .NET Mar 25: Cleveland C# User Group   Conferences Feb – Jun: Agent Camp Mar 16: Memphis Agent Camp  
Post a Comment
Read more

Databricks Q&A

-
Yesterday I delivered a presentation on "Data Cleansing using Databricks".  Listed below are the questions that came up as well as the answers related to Databricks.   Does state transfer from one cell to another in the Databricks notebook? Yes, state does transfer from one cell to another, as long as you're running in the same notebook session and on the same cluster.   Can Databricks Jobs (aka "pipelines") be accessed through a notebook? Yes, you can fully interact with Jobs from a notebook using the Databricks REST API or the Databricks CLI. Some of these interactions from a notebook include: Trigger a job run Check job run status Cancel a run Retrieve job metadata   However, you can't directly "open" the Jobs UI from a notebook, and you can't modify job definitions without using the API   Listed below is an example for accessing a Job from a notebook import requests import json   token = dbutils.secre...
Post a Comment
Read more

Layers of RAG Architecture Patterns

-
Retrieval ‑ Augmented Generation (RAG) has become one of the most important design patterns in modern AI because it gives language models direct access to external knowledge. Instead of relying solely on what a model has memorized during training, RAG systems retrieve relevant information from documents, databases, or other sources and feed it into the model at generation time. This idea dramatically improves accuracy, reduces hallucinations, and allows AI systems to stay current without constant retraining.   RAG has evolved into a rich ecosystem of architectural layers, addressing different challenges: Core Retrieval layer focuses on improving how information is found, from basic vector search to more advanced techniques like query expansion and hierarchical retrieval. Structure ‑ Aware layer organizes and interprets data based on relationships, formats, or time, enabling retrieval from graphs, tables, or multimodal sources. Reasoning ‑ Enhanced layer st...
Post a Comment
Read more

Linking one App.config in multiple projects

-
Image
I recently had a .NET solution where I wanted to use the same App.config in 2 different projects.  I wanted to always ensure that changes made in the App.config of the original project would be reflected in other project(s) automatically.  Listed below are the steps I used to facilitate that process. 1.      Right-click your project in Solution Explorer 2.      Select "Add" -> "Existing Item..." 3.      Navigate to the file that you want to add to the solution 4.      [Important]  Instead of hitting Enter or clicking the Add button, you want to click the down-arrow icon at the right edge of the Add button, and select "Add As Link".    
Post a Comment
Read more

Uploading documents to AI Foundry Agents

-
Image
Q: Can we upload documents like pdf as input and configure the agents to retrieve the required content that is expected   A: The short answer is yes, but with some configuration.  Working in the Agent Playground, you have the option to “Add” Knowledge.  This knowledge can be from a variety of different data sources as seen below. As stated below “Currently only a single instance per each type of data source is supported.”  In my scenario, I had a single text file setup when I configured my agent, before being published.   Another option would be to utilize the Azure AI Search to index multiple documents from a data store, where documents can be uploaded after the agent is published.  As of today, there are 6 other options available for accessing documents    
Post a Comment
Read more

AI Agents vs. Agentic AI

-
AI agents and agentic AI are related but not the same. AI agents are task oriented systems built around LLMs, while agentic AI refers to a broader paradigm where AI systems exhibit autonomy, goal directed behavior, and self improving capabilities.   AI Agents vs. Agentic AI AI Agents: perform tasks but do not necessarily set their own goals. •             Modular systems built around LLMs or LIMs. •             Designed for narrow, task specific automation. •             Operate through tool integration, prompt engineering, and workflow orchestration.   Examples: ·                   A customer service chatbot ·              ...
Post a Comment
Read more