DirectML, ML.NET, and CUDA in a nutshell

 

DirectML (Direct Machine Learning) is a low-level API for machine learning.  It was developed by Microsoft as part of its Windows AI platform. DirectML integrates with DirectX 12 compatible hardware. It’s designed to provide hardware-accelerated machine learning capabilities across a wide range of GPUs, not tied to any specific vendor.  DirectML is a low-level hardware abstraction layer that enables machine learning workloads on any DirectX 12 compatible GPU.

 

ML.NET (Machine Learning .NET) is an open source and cross-platform framework also developed by Microsoft.  It provides the capability to train and build custom machine learning models using C# or F#.

ML.NET also provides model building capabilities using various features:

1. AutoML (automated machine learning) that automatically attempts various algorithms to find the best fit for a given set of data
2. ML.NET CLI (Command Line Interface) for building models through the command line
3. Model Builder for building and integrating machine learning models into an application using a GUI interface.

 

CUDA (Compute Unified Device Architecture) is a parallel computing platform and programming model developed by NVIDIA for use exclusively on NVIDIA GPUs. 

It allows developers to use GPUs for deep learning and model building.

CUDA can be used with ML.Net Model Builder: https://learn.microsoft.com/en-us/dotnet/machine-learning/how-to-guides/install-gpu-model-builder

 

DirectML and CUDA are both frameworks used for machine learning and GPU computing, but they have distinct differences.

CUDA is generally faster for deep learning workloads, especially for complex models and large datasets. DirectML, while competitive, may not match CUDA's performance in high-end applications.

DirectML provides a high-level API, making it easier to use for developers. CUDA offers a low-level API, which allows for more fine-tuned control but requires more expertise.

DirectML is a good choice for cross-platform applications or when working with diverse hardware. CUDA is ideal for high-performance tasks on NVIDIA GPUs.

 

 

Phi Family of Small Language Models

Phi models are Small Language Models (SLM) developed by Microsoft.  They’re designed to handle various tasks, including text, image, and speech processing, while requiring less computing power.  The models are open-source, available with the MIT License.

 

The diagram below shows the evolution and capabilities of various Phi models. 

 

 

 

With the recent release of Phi-4 Multimodal model, more features are now available. In addition, here are some of its most notable features:

1. Multimodal Data Processing: Phi-4 Multimodal excels at handling text, images, and speech at the same time. This means it can interpret and generate content across different formats, making it incredibly versatile for various applications.

2. Efficient Performance: Despite its advanced capabilities, Phi-4 Multimodal is designed to be highly efficient. It requires significantly less computing power compared to larger AI systems, making it accessible and practical for a wider range of users and devices.

3. Enhanced Understanding: With its ability to integrate information from different data types, Phi-4 Multimodal offers a deeper and more comprehensive understanding of the context. This leads to more accurate and relevant responses, whether it's generating text, recognizing images, or interpreting speech.

4. Real-Time Processing: One of the most impressive features of Phi-4 Multimodal is its capability to process information in real-time. This is particularly beneficial for applications requiring instant analysis and response, such as virtual assistants, real-time translation, and interactive applications.

5. Customizability: Phi-4 Multimodal is designed with flexibility in mind. Users can tailor its functions and capabilities to suit specific needs, making it a highly customizable tool for developers and businesses.

 

For more info, please visit the Educator Developer Blog

For C# labs using Phi models, visit the PhiCookBook

 

Microsoft 365 Copilot Data Q&A

Question:  I need to know if data entered and used in the $30 Copilot service in M365 is secured in the same way that data in the $0 M365 Copilot Chat. I cannot find a reference that explains this. I want to know if my users can use both without the risk of having our content exposed outside of our tenant.

 

Answer: Yes, both the $30 Microsoft 365 Copilot service and the $0 Microsoft 365 Copilot Chat offer the same level of data security and privacy protections. Both services are covered by the same enterprise data protection (EDP) controls and commitments under the Data Protection Addendum (DPA) and Product Terms. Your data is protected with encryption at rest and in transit. Also, Microsoft does not use your data to train foundation models.  In a nutshell, your data remains yours and yours alone. 

For additional references, see the following links:

https://learn.microsoft.com/en-us/copilot/privacy-and-protections

https://learn.microsoft.com/en-us/copilot/microsoft-365/enterprise-data-protection

https://learn.microsoft.com/en-us/copilot/microsoft-365/microsoft-365-copilot-privacy

 

 

Question: What encryption methods are used to secure my data for Microsoft 365 Copilot?

 

Answer: There are multiple encryption methods are used to secure your data:

1. Encryption at Rest: Data stored in Microsoft 365 is encrypted using BitLocker and Distributed Key Manager (DKM). This ensures that your data is protected even when it is not actively being used.
2. Encryption in Transit: Data transmitted between your device and Microsoft 365 services is encrypted using Transport Layer Security (TLS). This helps protect your data from interception during transmission.
3. Sensitivity Labels and Azure Rights Management: Microsoft Purview sensitivity labels and Azure Rights Management provide an extra layer of protection by applying encryption and access controls to your data. This ensures that only authorized users can access sensitive information.

For more information, visit https://learn.microsoft.com/en-us/copilot/microsoft-365/microsoft-365-copilot-architecture-data-protection-auditing

 

 

Question: What other security features does Microsoft 365 Copilot have?

 

Answer: Microsoft 365 Copilot has several security features for data protection:

1. Advanced Threat Protection: Microsoft 365 Copilot includes advanced threat protection to detect and mitigate potential security threats.  Features like Safe Links and Safe Attachments, help protect against phishing and malware attacks.
2. Data Loss Prevention (DLP): DLP policies help prevent sensitive information from being shared inappropriately. These policies can be customized.
3. Identity and Access Management: Microsoft 365 Copilot uses Azure Active Directory (Azure AD) for identity and access management. This includes features like multi-factor authentication (MFA), conditional access policies, and identity protection to help secure user accounts and control access to resources.
4. Compliance and Auditing: Microsoft 365 Copilot provides comprehensive compliance and auditing. Features like eDiscovery, audit logs, and compliance reports help organizations meet regulatory requirements and monitor user activity.
5. Information Protection: Microsoft Purview sensitivity labels and encryption provide an extra layer of protection for your data, applied to documents, emails, and other content.
6. Oversharing Controls: Microsoft 365 Copilot includes controls to prevent oversharing of data in SharePoint sites and OneDrive.

For more information, visit Microsoft 365 Copilot security documentation.

 

DeepSeek R1 in a Nutshell

DeepSeek R1 is an advanced AI model developed by the Chinese startup DeepSeek AI. It has gained significant attention for the following reasons:

• Open Source, available to for use by anyone.
• Comparable Performance to OpenAI's GPT-4 and ChatGPT o1 models on various benchmarks.
• DeepSeek R1 was reportedly trained on 2,788 GPUs at a cost of around $6 million, significantly less than the estimated $100 million cost to train OpenAI's GPT-4.
• Excels in reasoning tasks and has been trained using large-scale reinforcement learning without supervised fine-tuning.
• Availability on platforms like Azure AI Foundry and GitHub, making it accessible for developers and researchers.
• DeepSeek R1's open-source nature and cost-effective training have made it a notable player in the AI community, challenging the notion that larger models and more data always lead to better performance.

 

To get started, DeepSeek R1 is now available via a serverless endpoint through the model catalog in Azure AI Foundry.

Also, check out the GitHub Models blog post, where you can explore additional resources and step-by-step guides to integrate DeepSeek R1 seamlessly into your applications.

In addition, customers will be able to use distilled flavors of the DeepSeek R1 model to run locally on their Copilot+ PCs, as noted in the Windows Developer blog post.

 

What is Automated Intelligence?

Automated Intelligence refers to the use of technology to automate repetitive, rule-based tasks that typically require minimal human intervention. This includes everything from data entry to workflow management and beyond. The goal of Automated Intelligence is to streamline processes, increase efficiency, and reduce the potential for human error.

 

Artificial Intelligence encompasses a broader scope, including machine learning, natural language processing, and more. AI is designed to simulate human intelligence and can perform complex tasks like understanding language, recognizing patterns, and making decisions based on data. AI systems can learn and adapt over time, improving their performance with more data and experience.

 

Automated Intelligence (AI) and Artificial Intelligence (AI). Although they share the same abbreviation, their applications and implications can differ significantly.

- Scope: Automated Intelligence focuses on automating specific tasks, while Artificial Intelligence aims to replicate human-like intelligence across a wide range of activities.

- Learning Capability: AI systems can learn and evolve, whereas Automated Intelligence typically follows predefined rules and processes without learning capabilities.

- Application: Automated Intelligence is often used for straightforward, repetitive tasks, while Artificial Intelligence tackles more complex and dynamic problems.

 

How Automated Intelligence Can Transform Business Processes:

1. Efficiency Boost: Automating routine tasks frees up employees to focus on higher-value activities, leading to increased productivity.

2. Consistency and Accuracy: By minimizing human intervention, businesses can reduce errors and ensure consistent output quality.

3. Cost Savings: Automation can reduce labor costs and streamline operations, resulting in significant cost savings over time.

4. Scalability: Automated processes can easily be scaled to handle increased workloads without the need for proportional increases in resources.

 

 

"Data Science with .NET and Polyglot Notebooks" By Matt Eland

In the fall of 2024, I had the opportunity to work with Matt Eland and be one of the editors for his book “Data Science with .NET and Polyglot Notebooks: Programmer's guide to data science using ML.NET, OpenAI, and Semantic Kernel”.  Matt is a very intelligent and knowledgeable data science developer and it definitely reflected in his work.  He walks the reader through step-by-step directions to demonstrate key concepts in data science, machine learning, as well as polyglot notebooks.  This was one of the rare books that I could hardly put down.  I urge you to pick-up a copy and upgrade your data science skills.

 

 

 

Recap of "How To Tune A Multi-Terabyte Database For Optimum Performance"

On October 29, 2024 at the GroupBy conference, I was moderator for Jeff Taylor's session "How To Tune A Multi-Terabyte Database For Optimum Performance"

The video is available at https://www.youtube.com/watch?v=9j51bD0DPZE

 

Listed below are some take aways and Q&As from his session:

 

Ideal Latency time:

20ms for IO

10ms for TempDB

 

Crystal Disk Mark is a simple disk benchmark software: https://crystalmark.info/en/software/crystaldiskmark/

 

What is the overhead of running these diagnostics (i.e. diskspd and Crystal Disk)?

No adverse effects during mid-day testing, but don't run it during a busy time.

It's best to test it during both busy and non-busy times

 

Mutlipath: multiple network cards between host, switch and SAN appliance

 

For tempdb storage, what's preferable?

Shared space on a disk pool with a lot of drives or dedicated pool with just 2 drives (raid 1)? all drives of the same type (NVMe). 

"Shared" means with other databases

Run in memory for newer versions

Raid10 will be fastest

keep tempDB separate from other DBs

 

By in memory tempdb, does that mean memory optimized tempdb metadata option? Yes

 

Jumbo Frames are 8192 when enabled, should be used for storage network to avoid issues

to transfer more across the network

 

 

Raid 5 is best for economy/performance combination on both SSD and conventional drives

 

RAID 5 for data? What about that write penalty overhead? Why not RAID 10 ? RAID 10 is best but RAID 5 will sufficiently perform but make sure you have enough memory for operational needs

 

Use New Mbsv3 series VMs from Azure

 

Would you consider local raid0 for tempdb? Yes, you can but RAID 1 so it's redundant so it stays live.

 

nvarchar: N for "National" characters for various foreign languages, 2+ bytes per character

 

Prefer to use INT instead of BigInt

 

Datetime2 (8 bytes) is preferred over Datetime(6-8 bytes)

 

Unicode size on disk is one thing, size when in cache is a worse problem

 

What do you think about using IFI (Instant File Initialization) for log file in 2022?  Recommended

 

Avoid Heap tables.  However, Markus Winand, author or SQL Performance Explained, shows some specialized cases where Heap is better.

See https://medium.com/@nhatcuong/sql-performance-explained-by-markus-winand-some-notes-after-the-first-read-1dde208f2fd7 for more info

Temporal Tables FAQ

I had the pleasure of presenting Temporal Tables to the Capital Area .NET User Group on December 10, 2024.  Some interesting FAQ arose from that meeting so I thought it would be good to share it on my blog for reference.

 

1. Is the historical table logged to the Transaction log in the same way a conventional table is?  Will we “see” the inserts in the Tran log the same way we see them for a normal table?

No

 

2. Are Temporal Tables available in Azure?

Yes, in Azure SQL Database and Azure SQL Managed Instance

 

3. Will Temporal Tables work with graph tables?

No, Node and edge tables can't be created as system-versioned temporal tables

Ref: https://learn.microsoft.com/en-us/sql/relational-databases/graphs/sql-graph-architecture?view=sql-server-ver16

 

4. What triggers the purge of SQL Server Temporal Tables?

A background task is created to perform aged data cleanup for all temporal tables with finite retention period.

ref: https://learn.microsoft.com/en-us/azure/azure-sql/database/temporal-tables-retention-policy?view=azuresql

 

5. Can I modify recs in the history table with versioning OFF? 

Yes

 

6. Can I alter the current table with versioning ON? 

Yes, alters both tables simultaneously

 

7. Does EF Code First Support Temporal Tables (TT)?

Yes, EF Core 6.0 and later supports:

• The creation of temporal tables using Migrations
• Transformation of existing tables into temporal tables, again using Migrations
• Querying historical data
• Restoring data from some point in the past

 

8. When are Temporal Tables NOT a good fit? 

For Static data where a user can't change a field.

The following industries rarely, if ever, are users allowed to delete any data

• Financial services (i.e. stock purchases, banking)
• Purchases
• GPS Location Tracking
• Fraud Detection(Location of purchases, IP/location of login)
• Tracking appointments (Resolve Customer service issues)
• Car Rental company, tracking owner and mileage when vehicle was in possession
• Hotel room rental
• Airline seat charts, who sat where in which flight

 

9. Are there other approaches are available for managing historical data in the temporal history table?

Yes, The following four methods are available:

1. Stretch Database
2. Table Partitioning
3. Custom Cleanup Script
4. Retention Policy

 

10. Can Temporal tables be used with partitioned databases?

Yes, with some limitations. If current table is partitioned, the history table is created on default file group because partitioning configuration is not replicated automatically from the current table to the history table.

 

11. Can the temporal table be placed in another DB?

No, History table must be created in the same database as the current table.

However, it can be placed in a different schema within the same database.

Also, Temporal querying over Linked Server is not supported.

 

12. Can it be used with Elastic DBs?

Yes

 

13. Can I alter table schema with sys versioning = on? 

No

 

14. Can I add additional fields to History table?

No, fields and field nullability must be identical

 

15. Is there automatic truncation of history table?

Yes, using HISTORY_RETENTION_PERIOD = 6 MONTHS

 

Questions on Copilot Data Privacy

Q: concerned about what Microsoft and / or the US government can do with the data for a custom copilot. I’ve looked at the Microsoft copilot documentation but I didn’t find anything that clearly states what Microsoft can and cannot do with data used in custom copilots, do you have any resources that you can share?

 

A: Microsoft posted info about this topic specifically at https://learn.microsoft.com/en-us/legal/cognitive-services/openai/data-privacy?context=%2Fazure%2Fcognitive-services%2Fopenai%2Fcontext%2Fcontext#see-also

 

In a nutshell:

Your prompts (inputs) and completions (outputs), your embeddings, and your training data:

• are NOT available to other customers.
• are NOT available to OpenAI.
• are NOT used to improve OpenAI models.
• are NOT used to improve any Microsoft or 3rd party products or services.
• are NOT used for automatically improving Azure OpenAI models for your use in your resource (The models are stateless, unless you explicitly fine-tune models with your training data).
• Your fine-tuned Azure OpenAI models are available exclusively for your use.

The Azure OpenAI Service is fully controlled by Microsoft; Microsoft hosts the OpenAI models in Microsoft’s Azure environment and the Service does NOT interact with any services operated by OpenAI (e.g. ChatGPT, or the OpenAI API).

 

 

 

Q: Does Microsoft have the same Data Privacy policy for Copilot studio as Azure AI Studio?  Is there similar documentation for custom copilots created in copilot studio?

 

A: It’s seeming like it’s the same.  After logging into Copilot Studio, browse to https://www.microsoft.com/licensing/terms/product/PrivacyandSecurityTerms/all .  From there, you can download the Data Protection Addendum from https://aka.ms/DPA. (see attached).  On P.5 it states:

 

Nature of Data Processing; Ownership

Microsoft will use and otherwise process Customer Data, Professional Services Data, and Personal Data only as described and subject to the limitations provided below (a) to provide Customer the Products and Services in accordance with Customer’s documented instructions and (b) for business operations incident to providing the Products and Services to Customer. As between the parties, Customer retains all right, title and interest in and to Customer Data and Professional Services Data. Microsoft acquires no rights in Customer Data or Professional Services Data, other than the rights Customer grants to Microsoft in this section. This paragraph does not affect Microsoft’s rights in software or services Microsoft licenses to Customer.

 

 

'Build Your Own Copilot" Resource Links

Interested in building your own Copilot using Azure AI Studio?  Listed below are some useful links:

• Azure AI Studio Architecture
• Quickstart: Create a project and use the chat playground in Azure AI Studio
• Tutorial: Deploy an Enterprise Chat web app
• AI Studio FAQ

 

May '24 Regional Tech Events

User Groups

• May 2: Twin Cities .NET User Group
• May 7: Ohio North Database Training
• May 8: Azure Cleveland
• May 16: GLUG.NET
• May 22: Cleveland C# User Group

 

Conferences

• May 3: Stir Trek
• Jun 27-28: Kansas City Developer Conference
• Jul 26: Cincy Deliver

 

Apr '24 Regional Tech Events

User Groups

• Apr 2: Ohio North Database Training
• Apr 4: Twin Cities .NET User Group
• Apr 10: Azure Cleveland
• Apr 18: GLUG.NET
• Apr 24: Cleveland C# User Group

 

Conferences

• May 3: Stir Trek

 

Trainers in ML.NET

Machine learning tasks like regression and classification contain various algorithm implementations. 

Some tasks may utilize the same algorithm, such as the SDCA algorithm in both Binary Classification and Regression tasks

 

In some cases, the problem you are trying to solve and the way your data is structured does not fit well into the current algorithm.

If so, consider using a different algorithm for your task to see if it learns better from your data.

 

A trainer identifies a single algorithm used for a single task (i.e. Trainer = Algorithm + Task).

Listed below is a summary of trainers available in ML.NET. For more info, see guidance on which algorithm to choose.

 

Trainer	Algorithm	Task	ONNX Exportable
SdcaLogisticRegressionBinaryTrainer	SDCA	Binary classification	Yes
SdcaNonCalibratedBinaryTrainer	SDCA	Binary classification	Yes
SdcaMaximumEntropyMulticlassTrainer	SDCA	Multiclass classification	Yes
SdcaNonCalibratedMulticlassTrainer	SDCA	Multiclass classification	Yes
SdcaRegressionTrainer	SDCA	Regression	Yes
AveragedPerceptronTrainer	Averaged Perceptron	Binary classification	Yes
LbfgsLogisticRegressionBinaryTrainer	L-BFGS	Binary classification	Yes
LbfgsMaximumEntropyMulticlassTrainer	L-BFGS	Multiclass classification	Yes
LbfgsPoissonRegressionTrainer	L-BFGS	Regression	Yes
SymbolicSgdLogisticRegressionBinaryTrainer	Symbolic stochastic gradient descent	Binary classification	Yes
OnlineGradientDescentTrainer	Online gradient descent	Regression	Yes
LightGbmBinaryTrainer	Light gradient boosted machine	Binary classification	Yes
LightGbmMulticlassTrainer	Light gradient boosted machine	Multiclass classification	Yes
LightGbmRegressionTrainer	Light gradient boosted machine	Regression	Yes
LightGbmRankingTrainer	Light gradient boosted machine	Ranking	No
FastTreeBinaryTrainer	Fast Tree	Binary classification	Yes
FastTreeRegressionTrainer	Fast Tree	Regression	Yes
FastTreeTweedieTrainer	Fast Tree	Regression	Yes
FastTreeRankingTrainer	Fast Tree	Ranking	No
FastForestBinaryTrainer	Fast Forest	Binary classification	Yes
FastForestRegressionTrainer	Fast Forest	Regression	Yes
GamBinaryTrainer	Generalized additive model	Binary classification	No
GamRegressionTrainer	Generalized Additive Model	Regression	No
MatrixFactorizationTrainer	Matrix Factorization	Recommendation	No
FieldAwareFactorizationMachineTrainer	Field Aware Factorization Machine	Binary classification	No
OneVersusAllTrainer	One Versus All	Multiclass classification	Yes
PairwiseCouplingTrainer	Pairwise Coupling	Multiclass classification	No
KMeansTrainer	KMeans	Clustering	Yes
RandomizedPcaTrainer	Randomized Pca	Anomaly detection	No
NaiveBayesMulticlassTrainer	Naive Bayes Multiclass	Multiclass classification	Yes
PriorTrainer	Prior	Binary classification	Yes
LinearSvmTrainer	Linear Svm	Binary classification	Yes
LdSvmTrainer	Ld Svm	Binary classification	Yes
OlsTrainer	Ols	Regression	Yes