User Groups
Conferences
Virtual User Group Meetings for Mar/Apr ‘22
Mar 24: “Predicting Flights with Azure Databricks” - https://www.meetup.com/NWVDNUG/events/284074689/
Mar 29: “Machine Learning in .Net” - https://www.meetup.com/Edmonton-NET-User-Group/events/284389088/
Apr 4: Ohio North Database Training- https://www.meetup.com/ohio-north-database-training/events/
Apr 13: Azure Cleveland - https://www.meetup.com/Azure-Cleveland-Meetup/
Apr 21: GLUG.Net - https://www.meetup.com/GLUGnet/events/
Apr 28: Cleveland C#/VB.Net User Group - https://www.meetup.com/Cleveland-C-VB-Net-User-Group
Virtual User Group Meetings
Microsoft’s Azure offers a great deal of features and capabilities. One of them is Cognitive Services, where users can access a variety of APIs to help mimic a human response. Some features include converting text to spoken speech, speech to text, and even the equivalent human understanding of a spoken phrase. These services are divided into 4 major categories, as seen below. Please note “Computer Vision” and “Custom Vision” sound very similar but their capabilities are different, as outlined in the “Vision” section below.
Decision
Language
Speech
Vision
Note: Model can be exported for use: https://docs.microsoft.com/en-us/azure/cognitive-services/custom-vision-service/export-your-model
Overview
Speech-to-text is one of the many cognitive services offered by Azure. All Azure cognitive services can be classified into one of four primary categories:
Each category contains a variety of services, with Speech-to-text categorized in the “Speech” category. It converts spoken language to text using either an SDK or web API call. Both methods will require a subscription key, obtained by a brief resource setup in the Azure portal. Speech-to-text can be configured to recognize a variety of languages as displayed at https://docs.microsoft.com/en-us/azure/cognitive-services/speech-service/language-support.
In addition, the source of the speech could be either live spoken words, or a recording.
How to Use Speech-to-text
Project Setup (using SDK)
Why you should use Speech-to-text?
Accessibility! Most applications depend on users not being vision impaired. However, this prevents a significant number of users from using an application due to their impaired vision. Certainly screen readers have been available in the Windows OS for nearly 2 decades. This allows any user to understand what is displayed on the screen. However, an impaired user will have difficulty interacting with the user interface (i.e. submitting info, filling forms, etc.). Thanks to Speech-to-text, users can now speak to the application and have the words dynamically translated to text in the application. This makes the application accessible to more users, as well as ADA and WCAG compliant.
Recently Vinod Kurpad delivered a presentation on using AI to build document processing workflows. The presentation slides can be found at https://on24static.akamaized.net/event/34/60/86/0/rt/1/documents/resourceList1636579433122/useaitobuilddocumentprocessingworkflows1636579401517.pdf
By using AI, several boxed fields (i.e. SSN) on a forma can be treated as 1 field. It allows dynamic entities, like tables, to recognize multiple rows in a single table with various column types.
Listed below are questions and answers discussed during the presentation:
Form Recognizer is available in Power Apps and has a receipt processing app in Power Apps. Licensing model in this scenario is via Power Apps. https://powerapps.microsoft.com/en-us/blog/process-receipts-with-ai-builder/
Not yet, Layout currently supports text, checkboxes, selection marks and reading order. Paragraphs, headers and more are on the roadmap.
Currently the only supported format is JSON. But there are a number of JSON to XML converters that should enable you to convert the output to XML
Reading order already supports multiple columns. Try it out :)
Form Recognizer receipts returns all line items, date, total etc. You can write validations and additional logics on top of the output as post processing. See here for all fields extracted from receipts - https://docs.microsoft.com/en-us/azure/applied-ai-services/form-recognizer/concept-receipt
It's on the roadmap for a preview in the first half of next year.
https://customers.microsoft.com/en-us/story/financial-fabric-banking-capital-markets-azure
When using Form Recognizer no pre-processing of the images are needed. It supports low dpi and various image qualities.
Yes, a key value pair can also be a value with no explicitly labeled key, but a span of text that describes the value.
Yes, Form Recognizer supports handwritten in the following languages - English, Simplified Chinese, French, German, Italian, Portuguese, Spanish. see all supported languages printed and handwritten here - https://docs.microsoft.com/en-us/azure/applied-ai-services/form-recognizer/language-support
https://docs.microsoft.com/en-us/azure/applied-ai-services/form-recognizer/language-support
Form Recognizer supports the following file formats - PDF (digital and scanned and multi-page), Tiff, bmp, jpg, png
Yes it does. No pre-processing needed. Please send the source document to Form Recognizer as is and Form Recognizer will do all the work for you.
Cognitive Services also includes speech to text and transcription APIs that you can combine for interviews.
Try creating a resource with a different name and see if that helps- the Studio is continually improving after the recent preview. Also send an email to formrecog_contact@microsoft.com
Form Recognizer includes signature detection capability, you can train a model to detect if a signature is detected or not.
The Form Recognizer features (APIs) build on top of OCR and handle digital, images and hybrid documents transparently.
The demo showed the specific document rendered in the browser, you can add a URL field to the index which is a pointer to the location of the original document
Not yet, on our roadmap but not available yet. Currently you should split the document into pages in such case and send the individual pages to Form Recognizer and then unite them in post processing. For invoices and line items tables that span multiple pages are supported.
Yes, Form Recognizer enables opening an FOTT project and Form Recognizer supports backward compatibility and you can use 2.1 release models in the new release also.
If you know the different form types you expect to receive, you should be able to train a model for each form and compose all the individual models into a single logical model. Sending a document to this logical model will result in the document being classified to determine the right model to extract the fields. The selected model is also returned as part of the response
No data can be anywhere and you can stream the document to Form Recognizer for analyzing or send a URL. Only your training data to train a custom model needs to be in a blob and only 5 documents are needed to get started with custom and train your own model
Max file size is 50MB currently.
The Studio and the underlying files handle multi-page labels and navigating across pages to label the data you want to extract.
To train a model all documents within a project and a model needs to be from the same type, same format. You can then compose several models into a model composed so you do not need to classify the document before sending it to Form Recognizer. You need 5 forms from the same type to train a model.
Form Recognizer text extraction supports print and handwritten and we just expanded handwritten from English to Simplified Chinese, French, Italian, Spanish, German, and Portuguese
Yes, Form Recognizer supports multipage documents.
Form Recognizer is the resource you need to create in the portal
The previous version of the labeling tool that's now superseded by the new Form Recognizer Studio -https://fott-2-1.azurewebsites.net/
Layout extracts tables as they appear on the document and will try to extract all rows and columns including indicate merged rows and columns spans. When labeling tables if the table changes and has a variety numbers or rows or columns you can label it as a dynamic table.
Yes, Form Recognizer supports handwritten text in the following languages - English, Simplified Chinese, French, German, Italian, Portuguese, Spanish. see all supported languages printed and handwritten here - https://docs.microsoft.com/en-us/azure/applied-ai-services/form-recognizer/language-support
Connect to the same blob storage and the project will open with all the data and labeling completed.
AI Builder which is a part of the Power Automate stack uses Form Recognizer and you should be able to use the same set of capabilities in AI Builder as well.
We suggest benchmarking it on your content but we believe and customers have validated that the handwritten OCR is world class for the supported languages, and yes Form Recognizer and OCR tech are heavily used in healthcare and financial verticals that involve processing handwritten text
When training a custom model if the variations are slight a key name changed, moved etc. than Form Recognizer should be able to extract, if you see that it misses than you can add a few documents with the variation to the model and improve the model. If the variation is big than you should train a model per year and then compose them to a model composed.
You can start with the documentation https://docs.microsoft.com/en-us/azure/applied-ai-services/form-recognizer/ . Also note that Form Recognizer supports a free tier which should allow you to test all the different features without paying for it
Yes! Form Recognizer supports a few different languages with an SDK, but the REST API is available for languages not supported by an SDK
Form Recognizer supports split cells, merged cells tables with no borders and complex tables. See more details on table extraction int he following blog https://techcommunity.microsoft.com/t5/azure-ai/enhanced-table-extraction-from-documents-with-form-recognizer/ba-p/2058011
If you mean concatenated or complex tables, then yes, we are continuously improving on extracting those. On the other hand, table extraction supports tables with or without physical lines.
Form Recognizer is integrating within Power Apps see Form Processing apps. In reference to another 365 domain can you please reach out to Form Recognizer Contact Us and we can try to assist
On October 28, 2021, Jeff Fritz (Microsoft) presented “What’s new in .Net 6” to the Cleveland C#/VB.Net User Group.
A recording of that presentation is now available here on my YouTube Channel.
Listed below is a brief summary of the key points from that presentation.
C#10
Blazor
.Net Release and LTS (Long Term Support) Schedule
Resources
Virtual User Group Meetings
Virtual Conferences
Thanks to all who attended the "Global AI Back Together" Event on October 19, 2021.
Keynote: https://www.youtube.com/watch?v=_pJ771eAPvA
Check-In (Swag & Azure Passes): https://checkin.globalai.community/
Workshops
The Azure Function that can see: https://workshops.globalai.community/the-azure-function-that-can-see/introduction
Train and deploy a PyTorch model using the Azure Machine Learning platform: https://workshops.globalai.community/azure-machine-learning/introduction
YouTube: https://www.youtube.com/channel/UCU4ffaIzhsvMr_cCt9kjQMw
All presentations from the event will be posted. Please subscribe to the channel to get notifications when new videos are posted.
Evaluation: https://forms.office.com/r/ctYxEL4fDE
This week’s series of AI articles is from Dinesh Asanka, an 8x Microsoft MVP in SQL Server. He has been working with SQL Server for more than 15 years, written articles and co-authored books. Dinesh is a presenter at various user groups and universities.
Virtual User Group Meetings
Virtual Conferences
Computer Vision and Custom Vision are 2 subsets of services provided by Azure Cognitive Services.
Computer Vision: Analyze content in images.
Custom Vision: Customize image recognition to fit your business needs.
When using the Object Detection Prediction API, the response returned from azure will be a JSON dataset using the following format.
public class PredictionResponse
{
public string id { get; set; }
public string project { get; set; }
public string iteration { get; set; }
public string created { get; set; }
public Prediction[] predictions { get; set; }
}
public class Prediction
{
public string tagId { get; set; }
public string tagName { get; set; }
public string probability { get; set; }
public BoundingBox boundingBox { get; set; }
}
public class BoundingBox
{
public string left { get; set; }
public string top { get; set; }
public string width { get; set; }
public string height { get; set; }
}
Each BoundingBox object in the response is represented graphically by the red boxes, as shown in the sample image below
In addition, listed below are some Gotcha’s to watch out for when working with Object Detection
There has been some confusion created with the release of .Net and .Net Core releases and how their support will be impacted in the near future. Listed below is a summary from the Microsoft Support Policy:
.NET and .NET Core refer to several technologies including the runtime, ASP.NET Core, Entity Framework Core, and more.
.NET Core and future versions of .NET provide Long Term Support (LTS) releases that get three years of patches and free support.
Patches to releases are compatible, which eliminates risk adversely affecting applications.
For details, see the .NET and .NET Core support policy.
| Version | Original Release Date | Latest Patch Version | Patch Release Date | Support Level | End of Support |
| .NET 5 | November 10, 2020 | 5.0.10 | September 14, 2021 | Current | 6 months after .NET 6 release (around May 2022) |
| .NET Core 3.1 | December 3, 2019 | 3.1.19 | September 14, 2021 | LTS | December 3, 2022 |
For better visualization of the .Net roadmap, see the timeline below for release dates of past, present, and future releases.
