ML Kit is a mobile SDK from Google that uses machine learning to solve problems such as text recognition, text translation, object detection, face\/pose detection, and so much more!<\/p>\n
The APIs can run on-device, enabling you to process real-time use cases without sending data to servers.<\/p>\n
ML Kit provides two groups of APIs:<\/p>\n
- \n
- \nVision APIs<\/em>: These include barcode scanning, face detection, text recognition, object detection, and pose detection.<\/li>\n
- \nNatural Language APIs<\/em>: You use them whenever you need to identify languages, translate text, and perform smart replies in text conversations.<\/li>\n<\/ul>\n
This tutorial will focus on Text Recognition<\/em>. With this API you can extract text from images, documents, and camera input in real time.<\/p>\n
In this tutorial, you\u2019ll learn:<\/p>\n
- \n
- What a text recognizer is and how it groups text elements.<\/li>\n
- The ML Kit Text Recognition features.<\/li>\n
- How to recognize and extract text from an image.<\/li>\n<\/ul>\n
Getting Started<\/h2>\n
Throughout this tutorial, you\u2019ll work with Xtractor<\/em>. This app lets you take a picture and extract the X usernames. You could use this app in a conference whenever the speaker shows their contact data and you\u2019d like to look for them later.<\/p>\n
Use the Download Materials<\/em> button at the top or bottom of this tutorial to download the starter project.<\/p>\n
Once downloaded, open the starter project in Android Studio Meerkat or newer. Build and run, and you\u2019ll see the following screen:<\/p>\n
Clicking the plus<\/em> button will let you choose a picture from your gallery. But, there won\u2019t be any text recognition.<\/p>\n
![\"Chosen](\"https:\/\/files.koenig.kodeco.com\/uploads\/2025\/05\/xtractor-no-text-recognition-650x293.png\")
<\/a><\/p>\nBefore adding text recognition functionality, you need to understand some concepts.<\/p>\n
Using a Text Recognizer<\/h2>\n
A text recognizer can detect and interpret text from various sources, such as images, videos, or scanned documents. This process is called OCR<\/em>, which stands for: Optical Character Recognition.<\/p>\n
Some text recognition use cases might be:<\/p>\n
- \n
- Scanning receipts or books into digital text.<\/li>\n
- Translating signs from static images or the camera.<\/li>\n
- Automatic license plate recognition.<\/li>\n
- Digitizing handwritten forms.<\/li>\n<\/ul>\n
Here\u2019s a breakdown of what a text recognizer typically does:<\/p>\n
- \n
- \nDetection<\/em>: Finds where the text is located within an image, video, or document.<\/li>\n
- \nRecognition<\/em>: Converts the detected characters or handwriting into machine-readable text.<\/li>\n
- \nOutput<\/em>: Returns the recognized text.<\/li>\n<\/ul>\n
ML Kit Text Recognizer segments text into blocks, lines, elements, and symbols.<\/p>\n
![\"\"](\"https:\/\/files.koenig.kodeco.com\/uploads\/2025\/05\/blocks-lines-elements-650x185.png\")
<\/a><\/p>\nHere\u2019s a brief explanation of each one:<\/p>\n
- \n
- \nBlock<\/em>: Shows in red, a set of text lines, e.g. a paragraph or column.<\/li>\n
- \nLine<\/em>: Shows in blue, a set of words.<\/li>\n
- \nElement<\/em>: Shows in green, a set of alphanumeric characters, a word.<\/li>\n
- \nSymbol<\/em>: Single alphanumeric character.<\/li>\n<\/ul>\n
ML Kit Text Recognition Features<\/h2>\n
The API has the following features:<\/p>\n
- \n
- Recognize text in various languages. Including Chinese, Devanagari, Japanese, Korean, and Latin. These were included in the latest (V2) version. Check the supported languages here<\/a>.<\/li>\n
- Can differentiate between a character, a word, a set of words, and a paragraph.<\/li>\n
- Identify the recognized text language.<\/li>\n
- Return bounding boxes, corner points, rotation information, confidence score for all detected blocks, lines, elements, and symbols<\/li>\n
- Recognize text in real-time.<\/li>\n<\/ul>\n
Bundled vs. Unbundled<\/h2>\n
All ML Kit features make use of Google-trained machine learning models by default. <\/p>\n
Particularly, for text recognition, the models can be installed either:<\/p>\n
- \n
- Unbundled: Models are downloaded and managed via Google Play Services.<\/li>\n
- Bundled: Models are statically linked to your app at build time.<\/li>\n<\/ul>\n
Using bundled models means that when the user installs the app, they\u2019ll also have all the models installed and will be usable immediately. Whenever the user uninstalls the app, all the models will be deleted. To update the models, first the developer has to update the models, publish the app, and the user has to update the app.<\/p>\n
On the other hand, if you use unbundled models, they\u2019re stored in Google Play Services. The app has to first download them before use. When the user uninstalls the app, the models will not necessarily be deleted. They\u2019ll only be deleted if all apps that depend on those models are uninstalled. Whenever a new version of the models are released, they\u2019ll be updated to be used in the app.<\/p>\n
Depending on your use case, you may choose one option or the other.<\/p>\n
It\u2019s suggested to use the unbundled option if you want a smaller app size and automated model updates by Google Play Services. <\/p>\n
However, you should use the bundled option if you want your users to have full feature functionality right after installing the app.<\/p>\n
Adding Text Recognition Capabilities<\/h3>\n
To use ML Kit Text Recognizer, open your app\u2019s build.gradle<\/em> file of the starter project and add the following dependency:<\/p>\n
\nimplementation(\"com.google.mlkit:text-recognition:16.0.1\") \nimplementation(\"org.jetbrains.kotlinx:kotlinx-coroutines-play-services:1.10.2\") \n<\/pre>\n
Here, you\u2019re using the
text-recognition<\/code> bundled version.<\/p>\nNow, sync your project.<\/p>\n
\nNote<\/em>: To get the latest version oftext-recognition<\/code>, please check here<\/a>.
\nTo get the latest version ofkotlinx-coroutines-play-services<\/code>, check here<\/a>. And, to support other languages, use the corresponding dependency. You can check them here<\/a>.\n<\/div>\nNow, replace the code of
recognizeUsernames<\/code> with the following:<\/p>\n\nval image = InputImage.fromBitmap(bitmap, 0) \nval recognizer = TextRecognition.getClient(TextRecognizerOptions.DEFAULT_OPTIONS) \nval result = recognizer.process(image).await() \n \nreturn emptyList() \n<\/pre>\n
You first get an image from a bitmap. Then, you get an instance of a
TextRecognizer<\/code> using the default options, with Latin language support. Finally, you process the image with the recognizer.<\/p>\nYou\u2019ll need to import the following:<\/p>\n
\nimport com.google.mlkit.vision.text.TextRecognition \nimport com.google.mlkit.vision.text.latin.TextRecognizerOptions \nimport com.kodeco.xtractor.ui.theme.XtractorTheme \nimport kotlinx.coroutines.tasks.await \n<\/pre>\n
\nNote<\/em>: To support other languages pass the corresponding option. You can check them here<\/a>.\n<\/div>\nYou could obtain blocks, lines, and elements like this:<\/p>\n
\n\/\/ 1 \nval text = result.text \n \nfor (block in result.textBlocks) { \n \/\/ 2 \n val blockText = block.text \n val blockCornerPoints = block.cornerPoints \n val blockFrame = block.boundingBox \n \n for (line in block.lines) { \n \/\/ 3 \n val lineText = line.text \n val lineCornerPoints = line.cornerPoints \n val lineFrame = line.boundingBox \n \n for (element in line.elements) { \n \/\/ 4 \n val elementText = element.text \n val elementCornerPoints = element.cornerPoints \n val elementFrame = element.boundingBox \n } \n } \n} \n<\/pre>\nHere\u2019s a brief explanation of the code above:<\/p>\n
- \nLine<\/em>: Shows in blue, a set of words.<\/li>\n
- \nRecognition<\/em>: Converts the detected characters or handwriting into machine-readable text.<\/li>\n
- \nDetection<\/em>: Finds where the text is located within an image, video, or document.<\/li>\n
- \nNatural Language APIs<\/em>: You use them whenever you need to identify languages, translate text, and perform smart replies in text conversations.<\/li>\n<\/ul>\n
![\"Chosen](\"https:\/\/files.koenig.kodeco.com\/uploads\/2025\/05\/xtractor-no-text-recognition.png\")
<\/a><\/p>\n