How to extract information from a form photo in C# .NET Framework using OCR

Today’s tutorial will cover how to set up form recognition using optical character recognition. This will allow us to specify specific field names to organize our OCR data and even recognize handwriting. Don’t worry though, this won’t take all day to get started. We will be employing an API, which will allow us to get set up in a matter of minutes.

First we need to install the API Client using the console of Package Manager:

Install-Package Cloudmersive.APIClient.NET.OCR -Version 2.1.4

With that done, simply call ImageOcrPhotoRecognizeForm:

using System;
using System.Diagnostics;
using Cloudmersive.APIClient.NET.OCR.Api;
using Cloudmersive.APIClient.NET.OCR.Client;
using Cloudmersive.APIClient.NET.OCR.Model;
namespace Example
{
public class ImageOcrPhotoRecognizeFormExample
{
public void main()
{
// Configure API key authorization: Apikey
Configuration.Default.AddApiKey("Apikey", "YOUR_API_KEY");
// Uncomment below to setup prefix (e.g. Bearer) for API key, if needed
// Configuration.Default.AddApiKeyPrefix("Apikey", "Bearer");
var apiInstance = new ImageOcrApi();
var imageFile = new System.IO.Stream(); // System.IO.Stream | Image file to perform OCR on.  Common file formats such as PNG, JPEG are supported.
var formTemplateDefinition = new Object(); // Object | Form field definitions (optional)
var recognitionMode = recognitionMode_example;  // string | Optional, enable advanced recognition mode by specifying 'Advanced', enable handwriting recognition by specifying 'EnableHandwriting'.  Default is disabled. (optional)
var preprocessing = preprocessing_example;  // string | Optional, preprocessing mode, default is 'Auto'.  Possible values are None (no preprocessing of the image), and Auto (automatic image enhancement of the image - including automatic unrotation of the image - before OCR is applied; this is recommended).  Set this to 'None' if you do not want to use automatic image unrotation and enhancement. (optional)
var diagnostics = diagnostics_example;  // string | Optional, diagnostics mode, default is 'false'.  Possible values are 'true' (will set DiagnosticImage to a diagnostic PNG image in the result), and 'false' (no diagnostics are enabled; this is recommended for best performance). (optional)
var language = language_example;  // string | Optional, language of the input document, default is English (ENG).  Possible values are ENG (English), ARA (Arabic), ZHO (Chinese - Simplified), ZHO-HANT (Chinese - Traditional), ASM (Assamese), AFR (Afrikaans), AMH (Amharic), AZE (Azerbaijani), AZE-CYRL (Azerbaijani - Cyrillic), BEL (Belarusian), BEN (Bengali), BOD (Tibetan), BOS (Bosnian), BUL (Bulgarian), CAT (Catalan; Valencian), CEB (Cebuano), CES (Czech), CHR (Cherokee), CYM (Welsh), DAN (Danish), DEU (German), DZO (Dzongkha), ELL (Greek), ENM (Archaic/Middle English), EPO (Esperanto), EST (Estonian), EUS (Basque), FAS (Persian), FIN (Finnish), FRA (French), FRK (Frankish), FRM (Middle-French), GLE (Irish), GLG (Galician), GRC (Ancient Greek), HAT (Hatian), HEB (Hebrew), HIN (Hindi), HRV (Croatian), HUN (Hungarian), IKU (Inuktitut), IND (Indonesian), ISL (Icelandic), ITA (Italian), ITA-OLD (Old - Italian), JAV (Javanese), JPN (Japanese), KAN (Kannada), KAT (Georgian), KAT-OLD (Old-Georgian), KAZ (Kazakh), KHM (Central Khmer), KIR (Kirghiz), KOR (Korean), KUR (Kurdish), LAO (Lao), LAT (Latin), LAV (Latvian), LIT (Lithuanian), MAL (Malayalam), MAR (Marathi), MKD (Macedonian), MLT (Maltese), MSA (Malay), MYA (Burmese), NEP (Nepali), NLD (Dutch), NOR (Norwegian), ORI (Oriya), PAN (Panjabi), POL (Polish), POR (Portuguese), PUS (Pushto), RON (Romanian), RUS (Russian), SAN (Sanskrit), SIN (Sinhala), SLK (Slovak), SLV (Slovenian), SPA (Spanish), SPA-OLD (Old Spanish), SQI (Albanian), SRP (Serbian), SRP-LAT (Latin Serbian), SWA (Swahili), SWE (Swedish), SYR (Syriac), TAM (Tamil), TEL (Telugu), TGK (Tajik), TGL (Tagalog), THA (Thai), TIR (Tigrinya), TUR (Turkish), UIG (Uighur), UKR (Ukrainian), URD (Urdu), UZB (Uzbek), UZB-CYR (Cyrillic Uzbek), VIE (Vietnamese), YID (Yiddish) (optional)
try
{
// Recognize a photo of a form, extract key fields and business information
FormRecognitionResult result = apiInstance.ImageOcrPhotoRecognizeForm(imageFile, formTemplateDefinition, recognitionMode, preprocessing, diagnostics, language);
Debug.WriteLine(result);
}
catch (Exception e)
{
Debug.Print("Exception when calling ImageOcrApi.ImageOcrPhotoRecognizeForm: " + e.Message );
}
}
}
}

As you can see, a wide degree of customization is possible. We can specify specific fields with formTemplateDefinition, activate handwriting recognition with recognitionMode, set up automatic preprocessing, and specify a language for improved accuracy. Here is what our output will look like:

{
  "Successful": true,
  "FieldValueExtractionResult": [
    {
      "TargetField": {
        "FieldID": "string",
        "LeftAnchor": "string",
        "TopAnchor": "string",
        "BottomAnchor": "string",
        "AnchorMode": "string",
        "DataType": "string",
        "TargetDigitCount": 0,
        "MinimumCharacterCount": 0,
        "AllowNumericDigits": true,
        "VerticalAlignmentType": "string",
        "HorizontalAlignmentType": "string",
        "TargetFieldWidth_Relative": 0,
        "TargetFieldHeight_Relative": 0,
        "TargetFieldHorizontalAdjustment": 0,
        "TargetFieldVerticalAdjustment": 0,
        "Ignore": [
          "string"
        ],
        "Options": "string"
      },
      "FieldValues": [
        {
          "Text": "string",
          "XLeft": 0,
          "YTop": 0,
          "Width": 0,
          "Height": 0,
          "BoundingPoints": [
            {
              "X": 0,
              "Y": 0
            }
          ],
          "ConfidenceLevel": 0
        }
      ]
    }
  ],
  "TableValueExtractionResults": [
    {
      "TableDefinition": {
        "TableID": "string",
        "ColumnDefinitions": [
          {
            "ColumnID": "string",
            "TopAnchor": "string",
            "AnchorMode": "string",
            "DataType": "string",
            "MinimumCharacterCount": 0,
            "AllowNumericDigits": true
          }
        ],
        "TargetTableHeight_Relative": 0,
        "TargetRowHeight_Relative": 0
      },
      "TableRowsResult": [
        {
          "TableRowCellsResult": [
            {
              "ColumnID": "string",
              "CellValues": [
                {
                  "Text": "string",
                  "XLeft": 0,
                  "YTop": 0,
                  "Width": 0,
                  "Height": 0,
                  "BoundingPoints": [
                    {
                      "X": 0,
                      "Y": 0
                    }
                  ],
                  "ConfidenceLevel": 0
                }
              ]
            }
          ]
        }
      ]
    }
  ],
  "Diagnostics": [
    "string"
  ],
  "BestMatchFormSettingName": "string"
}