Reading Text inside image
- Thread starter arajan37
- Start date
Sukesh Kumaru
Active Member
Hi,
For Coordinates go using action called get screen bounds option.
For Coordinates go using action called get screen bounds option.
Sukesh Kumaru
Active Member
Hi,
I think i.e not possible in Blue Prism.
I think i.e not possible in Blue Prism.
Hi arajan37,
I have checked the screenshot you attached, but wasn't able to identify whether its a table with a Web application or a Windows one.
Can you provide those inputs and based on that there could be more questions or the next course of possible action could be decided.
I have checked the screenshot you attached, but wasn't able to identify whether its a table with a Web application or a Windows one.
Can you provide those inputs and based on that there could be more questions or the next course of possible action could be decided.
its a table with web Application. I have tried to scan the table but am getting only blank rows and columns. actually, I want to search the image with that particular word. since its dynamic, it can come anywhere on the table.
I want to locate the image with text and need to get its position(row and column)
I want to locate the image with text and need to get its position(row and column)
Hi arajan37,
If it is a web application then I'm sure you would be able to spy the records (images in this case) of the table.
Use HTML mode and for now just spy any one image of the table (say the 1st one) and post here what is returned in the 'Html path' attribute in the Application Modeller.
If it is a web application then I'm sure you would be able to spy the records (images in this case) of the table.
Use HTML mode and for now just spy any one image of the table (say the 1st one) and post here what is returned in the 'Html path' attribute in the Application Modeller.
Hi arajan37,
Option #1:
Assuming that the image table is going to be at the same location on the page at all the times whenever the webpage is opened. The image containing the text can be anywhere within that table but the table should be at the same place and the size of all the images should be same. That is how Region Mode requires the data to be - in the same place in that same region. We have to use Region mode since that is the one that reads a text from an image. Here's the logic you could make use of if all of the above assumptions are true.
For instance you have a table with images of 4 rows and 4 columns-
- Spy every single image in Region Mode.
- Name them as you spy them. Give the name such that it has the row and column number in the name - eg; Image1_1 meaning Image of 1st row and 1st column
- After you spy the Images in Region mode, in your diagram use a Reader stage. In the Reader stage you will get an action called as 'Read Text with OCR'. In the 'Store In' field give a Text data item (say 'TextFromImage'). When your diagram flows through this Reader stage it will store whatever text is written on that image into the Text data item.
If there is no text on that Image then nothing will be returned in that Text data item.
Now use a Decision stage to compare the Text data item (TextFromImage) with the data item where you have stored the word "home". (You can even do a hard coded comparison with the word "home")
If these two match with the text comparison then it means this is your correct image. Now you already know from the element name (Image1_1) that this is 1st row and 1st column. So store 1 in a new data item called as Row and 1 in another data item called as Column.
If this is done for all the images you will match the text on each of the images and find the right one, if it exists.
Option #2:
Another option is if you are able to save or download the images then you can open it any of the Image Viewer on your machine and then read the text from the image using the same way (Region Mode) as above.
Also be noted that if you use AA mode on the image, the Application Modeller could return an attribute called as Value which would contain the path/url to the Image. See if you could make use of that.
Refer Post #8 in below link. This also has got the Business Object attached to the post.
http://rpaforum.net/threads/image.1132/post-3485
If you are not able to download the images of your webpage you can still open this BO and see how the Region mode is used to get the text from an image.
Post back with what you come up with.
Option #1:
Assuming that the image table is going to be at the same location on the page at all the times whenever the webpage is opened. The image containing the text can be anywhere within that table but the table should be at the same place and the size of all the images should be same. That is how Region Mode requires the data to be - in the same place in that same region. We have to use Region mode since that is the one that reads a text from an image. Here's the logic you could make use of if all of the above assumptions are true.
For instance you have a table with images of 4 rows and 4 columns-
- Spy every single image in Region Mode.
- Name them as you spy them. Give the name such that it has the row and column number in the name - eg; Image1_1 meaning Image of 1st row and 1st column
- After you spy the Images in Region mode, in your diagram use a Reader stage. In the Reader stage you will get an action called as 'Read Text with OCR'. In the 'Store In' field give a Text data item (say 'TextFromImage'). When your diagram flows through this Reader stage it will store whatever text is written on that image into the Text data item.
If there is no text on that Image then nothing will be returned in that Text data item.
Now use a Decision stage to compare the Text data item (TextFromImage) with the data item where you have stored the word "home". (You can even do a hard coded comparison with the word "home")
If these two match with the text comparison then it means this is your correct image. Now you already know from the element name (Image1_1) that this is 1st row and 1st column. So store 1 in a new data item called as Row and 1 in another data item called as Column.
If this is done for all the images you will match the text on each of the images and find the right one, if it exists.
Option #2:
Another option is if you are able to save or download the images then you can open it any of the Image Viewer on your machine and then read the text from the image using the same way (Region Mode) as above.
Also be noted that if you use AA mode on the image, the Application Modeller could return an attribute called as Value which would contain the path/url to the Image. See if you could make use of that.
Refer Post #8 in below link. This also has got the Business Object attached to the post.
http://rpaforum.net/threads/image.1132/post-3485
If you are not able to download the images of your webpage you can still open this BO and see how the Region mode is used to get the text from an image.
Post back with what you come up with.
Hi, I have tired with Region. I saved all images with specific name and tried to pass through the Reading stage. its extract the text as expected.
but the problem is everytime the rows and columns gets differ I am unable to use the same elements with a new page.
with AA, I am able to get URL of each image when I tried to open that URL it's opening to a different browser and I don't know how to download it from a browser.
sample url : http://rpahackathon.co.uk/images/matrix/28.jpg
but the problem is everytime the rows and columns gets differ I am unable to use the same elements with a new page.
with AA, I am able to get URL of each image when I tried to open that URL it's opening to a different browser and I don't know how to download it from a browser.
sample url : http://rpahackathon.co.uk/images/matrix/28.jpg
Hi arajan37,
the problem is everytime the rows and columns gets differ I am unable to use the same elements with a new page
What does this mean?
when I tried to open that URL it's opening to a different browser and I don't know how to download it from a browser.
You need not download the image. You can allow the link to open in the browser. Just Launch the Image URL just like any other URL, (if it becomes necessary you may need to Attach the Object to the browser window), then use the Region mode to get the text as above and then close the browser. Continue the same for every image link.
Post #8 in below link uses the Image editor of the machine. Instead of opening in editor in your case it is opening in a browser window. Refer the Business Object attached to the post.
http://rpaforum.net/threads/image.1132/post-3485
the problem is everytime the rows and columns gets differ I am unable to use the same elements with a new page
What does this mean?
when I tried to open that URL it's opening to a different browser and I don't know how to download it from a browser.
You need not download the image. You can allow the link to open in the browser. Just Launch the Image URL just like any other URL, (if it becomes necessary you may need to Attach the Object to the browser window), then use the Region mode to get the text as above and then close the browser. Continue the same for every image link.
Post #8 in below link uses the Image editor of the machine. Instead of opening in editor in your case it is opening in a browser window. Refer the Business Object attached to the post.
http://rpaforum.net/threads/image.1132/post-3485
Last edited:
Sukesh Kumaru
Active Member
Hello,
For this you have to use surface automation techniques.
Use Region mode and draw a region which is of width and height of one record and read using Read Text with OCR (Loop this activity by increasing X and Y co-ordinates by the required pixels to read line-by-line until it reaches end of the table),
You will see it in unstructured form, perform some string operations (to separate firstname, lastname and AccountID),
Finally,
send those table data line by line to your destination i.e, Defined Collection.
Good Luck.
Hi veena,
You can use Blue Prism's List Region functionality to achieve this.
This video explains getting the data from the table you have shown in the screenshot.
@VJR and @Sukesh Kumaru I was able to get the information from the table. My only problem now is that table is changing depending what the person searched. So the number of rows will also change. I don't want to be dependent on the number of rows I put on my collection. How can I count the number of rows that showed after I click "Search"?
Sukesh Kumaru
Active Member
Hello,
Number of table rows varies according to search, it's fine.
But, starting row of the table must be at some particular location every time, note this particular coordinates and draw a region and keep on increasing the pixels of size of single row, make use of start x and start y.
Assume that there are 10 rows, it keeps increasing the pixels and extracts the all 10 table rows,
here's the tricky part when it reaches to 11th row, but there's no 11th row right,
it reads null values into data items using decision stage check if it is null, if it is null stop it.
That's it.
Good Luck.
Last edited:
Hi veena,
You can select the entire table and then if certain rows at the bottom are blank those can be handled (skipped, remove empty rows, etc.) in the process/object diagram.
@VJR and @Sukesh Kumaru done. Thanks!