Running one process on two resource PCs

administrator

New Member
I am trying to run one process on two resource PCs simultaneously. The process has been designed to use a lock on files that can't be accessed simultaneously and uses a queue to get the items.

The only way I have gotten this to work is by duplicating the process and running the duplicated at the same time the two resource PCs. This seems like an incorrect workaround solution, as now I have to make fixes to two files that are duplicates of each other.

What is the correct way to run one process on two resource PCs, with the aim of cutting down processing time to half?
 

sahil_raina_91

Active Member
When will this post be approved?

The very purpose of having environment locking is to restrict simultaneous access to something.

Here are couple of approaches :

1. Remove lock, READ the excel file in READ MODE (multiple bots can read the file at same time in READ MODE)
This is assuming that the file data is not to be written or edited, ONLY reading the data from file is needed.
1.1 If editing the file in necessary, instead of editing the file directly, store the edited data in Work queue using SET DATA action, and update the file in one go with new data which can be fetched from queue.

2. Add all relevant data needed for processing in the Work Queue Item Data, to eliminate the need to open the file again.

You need to explain what exactly are you doing with those files, for us to propose a better solution.
 

Pete_L

Active Member
Creaete a separate process that loads the work queue from the input file. Run this process on only 1 bot. No need for environment locking in this scenario, since only 1 bot will need to access the input file. Once the work queue is loaded, you should be able to run the main process on multiple bots. Blue Prism will handle the locking and unlocking of work items for you automatically, so that only 1 bot at a time will work on a given work item in the queue. As sahil_raina mentioned, store any intermediate results of calculations etc back to the work queue via SET DATA during processing. If there is any reporting to be done at the end of the process, follow the same idea: a separate process for the report generation step, run on only 1 bot.

This is the way we do it in our organization. One bot loads the work queue, then multiple bots work the items in the queue, and 1 bot does the reporting at the end. We have had as many as 35 bots working a single process and work queue this way.
 
Good idea @Pete_L. I have a doubt on this.
The 1st bot will start with taking the input file and loading them to queue.
How will the 2nd or multiple other bots know that the queue is loaded oru when to start doing the main process? - can u explain this in stages in Bp.
And likewise how will one bot know when to generate that report?

If you could explain these with Process steps by step on high lvl, it would be very helpful.
 

Pete_L

Active Member
Put the 3 separate processes in a Schedule. First, run the process that loads the queue (assigned to only 1 bot), next run the main process (assigned to multiple bots), and finally, run the report process (assigned to only 1 bot).
 
Thati understand. How will the next bot know that the previous bot has finished its work. Like for example. How does the second bot know that items are added to queue by First bot @Pete_L
 

Sravya V

New Member
Schedulers have an option in which multiple tasks can be scheduled on completion of a Task. For example, on completion of task 1 (consider in this scenario - process that loads queue) , task 2 (process that works on queue items) will be started.

Hope this answers.
 

Jothin Georgi

New Member
@Bala Shunmugam: Suppose you schedule the queue process to run at 9 AM in the first bot machine M1 and say it takes an average time of 5 minutes to complete the process, Keep the schedule for the second bot machine M2 to run the Main process at around 9:10 AM. You can schedule the main process every hourly or minutely in M2 with a specific time period to avoid any schedule misses as per your requirement.
 

Pete_L

Active Member
Thati understand. How will the next bot know that the previous bot has finished its work. Like for example. How does the second bot know that items are added to queue by First bot @Pete_L

What I was referring to was putting all of the tasks on the same schedule. When you set up a schedule with multiple tasks, the 2nd task won't run until the first task is finished, and the 3rd task won't run until the 2nd task is finished (if you set up the schedule that way). When you set up a task within a schedule, you can assign any bot (or multiple bots) to run that task. So, task 1 loads the queue and is set up to run on only 1 bot (to avoid contention of having multiple bots trying to use the single input file). Task 2 is set up to run on multiple bots, and task 3 is set up to run the report on only 1 bot (again to avoid contention with multiple bots wanting to write to a single output file). Refer to the Scheduler guide on the BP Portal.

The 2nd bot does not care which bot loaded the queue. All it cares about is that there are work items in the queue for it to process.
 

Kamal

New Member
What I was referring to was putting all of the tasks on the same schedule. When you set up a schedule with multiple tasks, the 2nd task won't run until the first task is finished, and the 3rd task won't run until the 2nd task is finished (if you set up the schedule that way). When you set up a task within a schedule, you can assign any bot (or multiple bots) to run that task. So, task 1 loads the queue and is set up to run on only 1 bot (to avoid contention of having multiple bots trying to use the single input file). Task 2 is set up to run on multiple bots, and task 3 is set up to run the report on only 1 bot (again to avoid contention with multiple bots wanting to write to a single output file). Refer to the Scheduler guide on the BP Portal.

The 2nd bot does not care which bot loaded the queue. All it cares about is that there are work items in the queue for it to process.
Hi @Pete_L

Cant we do like first bot will just load the queue and it will be an Active queue assigned with multiple resources. So once the queue is loaded the main process will start executing in all the resources assigned. Once that is done in the same process at the end it will generate the report.

Second approach is to use environment locking so that only one bot will get the access to load the queue from the file. In this case we don't need to create a separate process to load the queue. Every thing will be done in the same process but multiple resources.

Kindly correct me if i am missing something or what is right/wrong in this approach.
 

Pete_L

Active Member
Your first paragraph is close to what I had explained. In your example, you will need environment lock on the input file so that only 1 bot has control of it. The only other difference between what you described and what I described is that you have the main process also generating the report. In practice, you will eventually find that you will need to run the report again (if, say, the user of the output report deletes it by mistake for example). In a case like that, you need the report generation to be in a separate process, since you won't want to run the entire automation again just to get the report -- especially if that automation makes updates to a system. Your 2nd approach is also completely fine, but it requires the additional complexity of environment locking, which is totally removed by having a separate Populate Queues process. How you do it is up to you and/or your organizational standards. In my organization, we separate processes as much as possible as our standard.
 

Kamal

New Member
Your first paragraph is close to what I had explained. In your example, you will need environment lock on the input file so that only 1 bot has control of it. The only other difference between what you described and what I described is that you have the main process also generating the report. In practice, you will eventually find that you will need to run the report again (if, say, the user of the output report deletes it by mistake for example). In a case like that, you need the report generation to be in a separate process, since you won't want to run the entire automation again just to get the report -- especially if that automation makes updates to a system. Your 2nd approach is also completely fine, but it requires the additional complexity of environment locking, which is totally removed by having a separate Populate Queues process. How you do it is up to you and/or your organizational standards. In my organization, we separate processes as much as possible as our standard.
Thanks @Pete_L... Its a nice discussion.
One more thing i would like to know from you is:

Could you please explain how to separate our tasks ? Like, how to separate our tasks among Processes-Pages-Objects-Actions. From a big process which task should be done at which level. How to divide our actions inside an object. What should basically an object contain ? How to divide pages inside a process? And how should we divide processes ?
 

Pete_L

Active Member
As developers, we naturally think only of the requirements, and not of how things will work in practice once the automation is in production. Once in production, chances are that you will encounter scenarios that were not considered when the requirements were written, primarily because requirements usually only consider what needs to be accomplished and they do not always take into account what might or could happen once in production. I personally have a lot of business experience in addition to my experience as a developer. As a result, I try to develop my automations in a way that anticipates scenarios that are not contained in the requirements. An example of this is the example I gave of having my Populate Queues process always be in a separate process, and my Write Report process always to be separate too. This is not to say that those tasks cannot be accomplished using a single process, but real-world experience tells me that there will often be times when an issue arises that is best resolved by having things separated.

So, how to decide what gets its own process. I've already mentioned Populate Queues and Write Report. I always separate them. Other things that should be separated will depend on the requirememts of the automation, so it is difficult to say what you should do. As a rule, I separate processes that can logically be separated based on the system(s) they will interact with, the need to pass data between the processes, and the timing of how the steps need to be run. For example, if you want to separate two processes, and the 2nd process needs to consume data from the 1st process, they can be separate if the 1st process is set up to store its data back to the work queue and the 2nd processes gets its data from the work queue. Those processes can be separated. Or, if a process interacts with separate applications (like Excel and Word for example), then it might make sense to separate them into 2 processes. If your automation will interact with the 2 applications multiple times during the process, however, then it might make better sense to keep them together. I could offer many examples, but the discussion would go on for hours. Just think about what you need to accomplish, the applications you will use, and the timing of how things need to fit together and eventually you will get it.

How to separate pages in a Process. A process represents a start-to-finish task. During the course of this overall task, you will find that there are some things that are done multiple times. For example, let's say that your task must do some processing, and then it will post an entry to some system, like an accounting ledger or a sales system. The steps for posting that entry should be on a separate page, and that page should be referenced in the process when it is time to do that task. This allows the posting task to be reused, and makes maintaining it easier in the future. You pass data between the pages using inputs and outputs. Hopefully, this example gives you and idea of the concept.

In an Object, each page should represent an interaction with an application. So, if you have to log in to that application, you will have a login page. If you have to then go to a Main Menu, you would have a page called something like "Navigate Main Menu" and that page will contain the steps for going to the Main Menu. If you then have to select an item from the Main Menu (like, say, Create an Order), then you would have a page that contains the steps for selecting Create an Order from the Main Menu. Maybe you will have another page for the steps needed to Enter an Order. Each page in the Object should contain the steps for interacting with the application screen in some way, and nothing else. Each of these pages is called an Action. Then, as an example, in your Process when you need to login to the application, you use the Login action and when you need to Create an Order you use that action. I hope this example helps you get the concept.

I apologize for the lengthy post, but it is difficult to describe a concept without first setting up a scenaro and then giving guidance.

I hope this helps.
 
Last edited:

Kamal

New Member
As developers, we naturally think only of the requirements, and not of how things will work in practice once the automation is in production. Once in production, chances are that you will encounter scenarios that were not considered when the requirements were written, primarily because requirements usually only consider what needs to be accomplished and they do not always take into account what might or could happen once in production. I personally have a lot of business experience in addition to my experience as a developer. As a result, I try to develop my automations in a way that anticipates scenarios that are not contained in the requirements. An example of this is the example I gave of having my Populate Queues process always be in a separate process, and my Write Report process always to be separate too. This is not to say that those tasks cannot be accomplished using a single process, but real-world experience tells me that there will often be times when an issue arises that is best resolved by having things separated.

So, how to decide what gets its own process. I've already mentioned Populate Queues and Write Report. I always separate them. Other things that should be separated will depend on the requirememts of the automation, so it is difficult to say what you should do. As a rule, I separate processes that can logically be separated based on the system(s) they will interact with, the need to pass data between the processes, and the timing of how the steps need to be run. For example, if you want to separate two processes, and the 2nd process needs to consume data from the 1st process, they can be separate if the 1st process is set up to store its data back to the work queue and the 2nd processes gets its data from the work queue. Those processes can be separated. Or, if a process interacts with separate applications (like Excel and Word for example), then it might make sense to separate them into 2 processes. If your automation will interact with the 2 applications multiple times during the process, however, then it might make better sense to keep them together. I could offer many examples, but the discussion would go on for hours. Just think about what you need to accomplish, the applications you will use, and the timing of how things need to fit together and eventually you will get it.

How to separate pages in a Process. A process represents a start-to-finish task. During the course of this overall task, you will find that there are some things that are done multiple times. For example, let's say that your task must do some processing, and then it will post an entry to some system, like an accounting ledger or a sales system. The steps for posting that entry should be on a separate page, and that page should be referenced in the process when it is time to do that task. This allows the posting task to be reused, and makes maintaining it easier in the future. You pass data between the pages using inputs and outputs. Hopefully, this example gives you and idea of the concept.

In an Object, each page should represent an interaction with an application. So, if you have to log in to that application, you will have a login page. If you have to then go to a Main Menu, you would have a page called something like "Navigate Main Menu" and that page will contain the steps for going to the Main Menu. If you then have to select an item from the Main Menu (like, say, Create an Order), then you would have a page that contains the steps for selecting Create an Order from the Main Menu. Maybe you will have another page for the steps needed to Enter an Order. Each page in the Object should contain the steps for interacting with the application screen in some way, and nothing else. Each of these pages is called an Action. Then, as an example, in your Process when you need to login to the application, you use the Login action and when you need to Create an Order you use that action. I hope this example helps you get the concept.

I apologize for the lengthy post, but it is difficult to describe a concept without first setting up a scenario and then giving guidance.

I hope this helps.
Thank you so much @Pete_L. Its clear now.
 
Top