AI Builder
AI Builder – Form Processing
Category: AI Builder,Power Platform Author: Manonmani V S Date: 12 months ago Comments: 0


AI Builder brings in the below capabilities which would have taken more time and effort without the good news that the AI Modeling is now available in a no-code/low-code PowerPlatform that is gaining more traction:

  1. Binary Classification
  2. Form Processing
  3. Object Detection
  4. Text Classification

Want to know more about AI Builder? You may read here…

In this blog, I will show you the Form Processing feature. Let me explain quickly on how to read invoice data on a Canvas App using trained models in AI Builder.

First, let us create a model using AI builder and train it with sample invoices. Next, we will be using this model in a Canvas App to process the images.

Create a Model in AI Builder

  1. Select Form Processing – Log-in at and navigate to AI Builder > Build and select “Form Processing”. In the next screen, name your model and click “Create”.

2. Add documents – you will be given an option to upload documents. Select min 5 invoice images to train the model. The more images the better the results.

3. Analyse documents – Below screen shows the list of document for the model to Analyse. In this step the model will identify the data fields available on the document.

4. Select form fields – This step allows you to make a decision on what fields will be captured from the invoice to be processed in your system later. Click on the Image to select the fields.

On the right hand side, you will be able to see “All fields” as analysed by the model in the previous step. You may wish to select the fields that is required for your App and click “Next”.

5. Train your model

6. Quick Test your model – Navigate to Models>select your model and do a “Quick Test” by selecting a new invoice.

Yay! the model is trained to read the key-value pairs. The model has identified the below fields as highlighted in green. Mouse over to see the attribute key and value.

7. Publish – Now Publish it to make it available for Canvas App.

Create a Canvas App for Form Processing

  1. Create App – click on “Create app” and choose “Canvas App”. Give your App a name and save it.

2. Navigate to Insert>AI Builder>FormProcessor

3. Select the model that you have trained earlier.

4. You will see the Form Processor AI Component. Run the preview F5

5. Click “Analyse” to add a new invoice that you wish to process. The app will identify the fields that we have trained the model for.

6. How to Extract invoice data to the Canvas App form? It is pretty simple as using the below formula.


You may now save this data to trigger flows.

Further reading…†



Source: Mano

AI Builder – Object Detection: walkthrough
Category: AI Builder,Power Apps,Power Platform,PowerApps Author: Jeevarajan Kumar Date: 12 months ago Comments: 0

If you had been watching the live streaming of Microsoft Business Application Summit, 2019 kick off and keynote sessions by James Phillips, Wim Coorevits and others. I’m pretty sure you are stunned as I am. Out of all those astonishing releases, the AI builder is something that fell in love immediately and I wanted to try it out right away, couldn’t believe how the AI modelling is wrapped up and made simple for the Citizen Developers to use it.

Before AI modelling, the same set of features would have taken a considerable amount of time to build the same with PowerApps + Flow + Azure Cognitive Services.

I just tried and I explain here about creating an Object Detection AI Model of the AI Builder to identify my perfume, couldn’t think too much at 3 A.M :). Before jumping into the steps, let’s see a quick overview of what is AI builder and some real-time use cases.

For a quick demo, check out my LinkedIn post:

AI Builder is a low code artificial intelligence platform that supports the Power Platform. It will be available for consumption on data that already exists in the Common Data Service (CDS), the enterprise-grade datastore included in the Power Platform

Key capabilities of AI builder as of now:

  1. Binary Classification – uses historical data to predict whether new data falls into one of two categories. AI Builder binary classification is an AI model that predicts yes/no business outcomes by learning to associate historical data patterns with historical outcomes. 
  2. Text Classification – one of the fundamental Natural Language Processing (NLP) problems. It allows tagging of text entries with tags or labels which can be used for sentiment analysis, spam detection and routing customer requests, just to name a few examples.
  3. Object Detection – lets you count, locate, and identify selected objects within any image.
  4. Business Card Reader – is a component available in the PowerApps studio that lets you scan business cards. You can use this control to extract contact information from pictures of business cards or your mobile phones camera.
  5. Form Processing – identifies the structure of your documents based on examples you provide to extract text from any matching form. Examples might include tax forms or invoices

Make sure your admin has enabled AI Builder for your environment. Otherwise, you won’t have access to the AI builder functionality. More information: Enable or disable AI Builder feature. 

1. Navigate to Build-> Objection Detection

1 - a

2. Choose the object name [Buid  a Model]

I Builder requires the use of Common Data Service, which is the data platform for PowerApps and allows you to store and manage business data. Common Data Service is the platform on which Dynamics 365 apps are built so if you’re a Dynamics 365 customer, your data is already in Common Data Service.



Choose the record you would like to play with. I wanted to build a model that can identify Versace Eros and Burberry London, so I select them both.1-1

3. Collect images [Train a Model]

Upload at least 15 images to proceed further. Microsoft recommends at least 50 images per product for the training set as a starting point. With fewer images, there’s a strong risk that your model will learn concepts that are in fact noise or irrelevant details. Training your model with more images will generally increase the accuracy of prediction results.


It prevents uploading of duplicate image (hope its not by the name).


Rejects if the image doesn’t solve the minimum requirement


4. Tagging images 

Navigate from image to image, and tag at least 15 images per object name to build a model.


Tagging requirements pane shows the objects being tagged and their tag counts.


5. Train the Model


Status can be tracked  under the Models pane


6. Test & Publish: 

Use the ‘Quick Test’ to test the model’s accuracy and then Publish to consume it in the app.


Publishing the model:


The app will throw below error if the referenced model is unpublished.


Status is changed to ‘Live’ once published


Either you can create an app using ‘Create app’ or can be inserted from the AI Builder (preview control).


7. Using the model in the App

AI Builder controls as shown below


I have added a gallery and used the ObjectDetector.VisionObjects result to bind the Name and Count.


Hope this would be a starting point to explore the AI Builder. 

Source: Jeeva