Activiti docker image

Docker is a fantastic virtualization tool for running different images very fast and with minimum effort. There is a hub available with many pre-created ready to use images.

With docker it is matter of minutes to download and run images for Activiti explorer and rest Api.

It would be greatly helpful for people who want to play and evaluate the tool. Also for people who make demos and present Activiti.

First you need to install docker, if it is not already done. Installation is very straight forward. Binaries are available for nearly every major OS.

After installation, you need to pull the image:

docker pull smirzai/activiti-single-image

Downloading image may take some time, depending on your internet connection. After that you can easily run explorer or Rest-API.

Executing this command runs the explorer in the container:

sudo docker run -p 8080:8080  -t -i smirzai/activiti-single-image:latest explorer

After less than a minute, container will be running. You can access the explore using such an URL in your explorer:

http://localhost:8080/activiti-explorer

voilà ! you are there.

Similarly you can run Rest API, using such a command:

sudo docker run -p 8080:8080  -t -i smirzai/activiti-single-image:latest rest

Explorer is the default, if you omit the last parameter.

We are working on making it official out of my personal docker hub.

 

 

scip docker image

SCIP  seems to be a wonderful open source tool for solving constraint integer problems. It is the standard tools for many university courses. The problem is the installation is not trivial specially for pure mathematicians.

Docker comes to rescue. After straight forward installation of docker, you need only these few steps to run scip:

 

docker pull smirzai/scip

docker run -i smirzai/scip

scip

 

you are there. you can use scip now. The only catch is that you have to send your files to the container using  docker ps command.

 

PS: I was contacted by SCIP team and they kindly advised me to remove the docker image, as user does not need to accept license agreement with the current installation. I try to solve the problem with them and put the image back in docker hub. For now I have to remove it.

 

problems of confluence as document writing tool

Here is the list of problems I faced in using confluence for writing documents and then converting them to word or PDF.

1. Master/Slave versioning:

if you have a master document which consists of several child pages, each of them will have their own version history. consequently you cannot use versioning that comes with confluence for parent/child pages, unless you are happy with having several change history tables, with different version numbers in the final document.

2. Offline usage
You cannot work on your documents where you are on a train.

3. Loosing content
Ok, this is only one time experience. After half an hour working on documentation, I got a horrible error message “you have enough credentials” and all my work vanished.

4. source control
you cannot push your documents in source control. This means that you have to keep the source and the documentation in sync manually.

5. fine grained versioning
I don’t really want to increase the version each time I change a comma in the text. As far as I know there is no tool, to let me decide when version should be incremented.

6. hard editing
If I find a typo in my docs, which I often do, and I want to correct it, I need to scroll up to the top of the page, press edit button (this can be done faster by keyboard shortcut), scroll back and find the typo and correct it, then I need to scroll down to the bottom of the page and press save (For the last part I think ctrl-s is a keyboard shortcut). This can be cumbersome in long documents.

Activiti Maven Archetype

As activiti developer, It happens too often that one has to create sample test cases . Some boilerplate code is necessary even for the most basic test case to run. Things like sample config file, sample process model, test case.

To make it faster, Joram has created a “Activiti Unit Test template” which makes life much easier. Personally I have used it a lot. It does most of the job, but again you need to do some customization to fully use it. To say, if you use multiple copies, without change, you can import only one of them in Eclipse.

There are tools for scaffolding and generating which allow faster, more dynamic and customizable templates. As maven is already used, maven archetype seems to be the logical tool.

In the most recent version of activiti (version 5.17.1) , the  first activiti maven archetype contains the same code written by Joram, in a new shell.

The archetype will be available in standard activiti maven repository. This simple command, will create a unit test template.

mvn archetype:generate
-DarchetypeGroupId=org.activiti.archetypes
-DarchetypeArtifactId=activiti-archetype-unittest
-DarchetypeVersion=5.17.1-SNAPSHOT
-DgroupId=org.myGroup
-DartifactId=myArtifact

 

The first three parameters  specify the archetype to be used. GroupId and artifactId determines your customized groupId and artifactId. Packages will be created according to the given group Id.

As simple is that.

This is just the start. Lot’s of extensions are possible. Ideas like creating a template for a web application (one with angular.js and one with jsf), selecting database type and so on.

 

Real Multithreaded parallel execution in activiti

Here is a real scenario:

We are going to implement a  web service using activiti. So we should be synchronous and quick. In this scenario call comes from Camel endoint, but it is not relevant for this discussion and we can ignore it safely without loosing generality.

To conclude the service, we need to call 3 other external web services. External web services take random amount of time to return back the result. After all 3 web services are returned, we need to aggregate their returned results and return the result of aggregation as the return value of our web service.

This diagram shows the outline:

parallelSampleSerial

 

This works fine. As each step is synchronous, activiti executes the next step only when the previous one is returned. The returned values is stored as a process variable. At aggregation we have all the returned values and we can aggregate.

But if the web service calls are independent, we can run them in parallel and save time and resources. And here is where all the problems begin.

BPMN has already a parallel gateway available. The first temptation is  using that feature. something like this:

parallelSampleParallel

 

It looks nice on the paper, but we know that at activiti uses a single thread to run all the three paths. This means that at the end of the day, they are sequenced. There is no differences to the previous one, except that we cannot be sure about the order of the execution of web service calls. Not very promising !

What if we make the parallel services asynchronous? Ok, that is not easy as well. Here is a nice post from Tijs about how it could be done. It might be a little outdated though, thank to the new Async Job Executor starting from version 5.17.0.

With this approach, aggregate part is done properly. Activiti makes sure, the aggregate service is called only when all 3 web service calls are finished. good !

But the problem is that we need to return aggregated result as the return of web service. As the parallel tasks are asynchronous, the control returns to the caller immediately, without waiting for the external web service calls to return. So we don’t have the result yet. We need to wait and synchronize for the result to come. Usual way to do it is by locks and monitors in Java. Not exactly feasible here, as the the very same objects are not passed to the process. They are serialized and deserialized  via database.

To solve that problem, we can maintain a global static array of objects, and only pass the reference to the proper lock object to the activiti. In aggregation service, the static object is found and signaled with the help of passed index. Signaling causes the waiting thread to release and result to return back to web service.

Just as a rough proof of concept there I have created a  sample, available  in   parallelMultiThreaded in github.

In the sample MyUnitTest test class simulates the body of newly created web service call. Service1, Service2, Service3 simulate external web service calls, simply by sleeping. The executed model is the second one above with parallel gateway.

The static lock and condition arrays are defined as:

 public static Condition[] conditions = new Condition[1000];
 public static Lock[] locks = new Lock[1000];

 

index, is given a random number (0, not much random I confess) and passed to the activiti.
After starting the instance, main code goes to an unlimited wait loop, waiting for the corresponding object to be unlocked:

condition.await();

After all the external web services are executed, activiti executes aggregation task, which in turn signals the lock:

condition.signalAll();

The log files clearly show that everything goes fine:

02:35:16,817 [pool-1-thread-1] INFO  org.activiti.Service1  - Service 1 started, time = 70
02:35:16,817 [pool-1-thread-2] INFO  org.activiti.Service2  - Service 2 started, time = 70
02:35:16,817 [pool-1-thread-3] INFO  org.activiti.Service3  - Service 3 started, time = 70
02:35:17,318 [pool-1-thread-3] INFO  org.activiti.Service3  - Service 3 finished, time = 571
02:35:17,519 [pool-1-thread-2] INFO  org.activiti.Service2  - Service 2 finished, time = 772
02:35:17,819 [pool-1-thread-1] INFO  org.activiti.Service1  - Service 1 finished, time = 1072
02:35:17,859 [pool-1-thread-1] INFO  org.activiti.AggregateService  - All external ws calls returned. Try to unlock the lock. time = 1112
02:35:17,859 [main] INFO  org.activiti.MyUnitTest  - wait loop exited. Program finished. total taime spent = 1112

It is clear that tasks are running simultaneously  and aggregation and monitoring are done properly. Also there is no considerable delay for unlocking and signaling.

 

Ok, I am cheating. The time for sleeping are intentionally set to different values with proper distance. Otherwise contention will cause Asynchronous job executor, to retry. This adds several seconds to the whole process. But that is not the subject of this post.

The sample code is not considering optimistic locking problem as is described in the post above, neither it handles timeout in waiting for lock to be release,  but well shows the concept. So with this workaround, it is possible to use activiti parallel gateway for executing external web service really multi-threaded and respond the result synchronously.

But one may suggest to overlook activiti parallel gateways, and do the thread synchronization in a single Java delegate. That way it would be much  conciser and easier to implement. It is much faster, as it does need communicating via database. The cons is that you can not see the real flow in the diagram.

 

Obviously this only works when everything is running in a single jvm. It is possible to implement it in a cluster, but even the current implementation is not much practical to the long delay caused by asynchronous job picking mechanism

PS: Thank to Joram’s hint, I replaced the low level wait and notify to higher level ReenterantLock.

 

 

 

Exporting diagrams from Bizagi to Activiti Designer

Activiti comes with two modelers, eclipse based and web based. Activiti does not promise NO-CODE workflow and modelers are also designed having a developer as user in mind, specially the eclipse one. One can easily forget about eclipse based designer when trying to use it to discuss a business process with a domain expert or a business analyst. They need some more graphical tool than eclipse.

Bizagi is a robust clean free tool for such purposes. One can use it easily to draw a conceptual BPMN diagram and even simulate it in some extend.

But what if you decide to execute the model using Activiti ? In older versions, it was not possible to reuse the diagram available in Bizagi. The whole process diagram needed to be recreated in activiti from scratch, blank paper style. The most recent version (I am using version 2.5 for this test), supports exporting BPMN diagram to various formats including BPMN. This format is what activiti uses by default. So theoritically it should be possible to open the diagram created in Bizagi  inside eclipse based designer and edit it there.

But they do not funtion always as advertised. I did a short test to check how possible it is.

First of all, it is important to notice the two tools serve more or less different purposes. Activiti designer cover all the detailed required information to really execute the model by its own engine. Diagram in Bizagi is more a conceptual diagram and lack those details. So obviously in best case, the converted diagram is not executable. Only the base structure will be converted and all the details, class named, error codes, and so on should be added manually. Not ideal, but way much better that redrawing every thing from scratch.

Exported diagram can be opened in Eclipse designer and usually the basic patter nis readily visible. It is fine as long as you have not tried to edit something. The smallest change and save makes the whole disagram be unreadable the next time. The reason is that for any reason activiti does not understand the Collaboration semantics produced by Bizagi. Whose fault is it, should be checked and fixed. Activiti designer only uses it for graphical representation after all, no effect on the execution.
So one workaround is to delete the whole content of <Collaboration> tag as first step before trying to save the diagram.

The above trick makes the diagram editable in eclipse designer. Most of the commonly used BPMN elements are compatible from Bizagi to Activiti, but not all of them. I did a quick test on what is supported and what is not and here is the result:

UnSupported BPMN Elements:

  1. Subprocess
  2. CallActiviy
  3. Shell Task
  4. Camel Task
  5. Mail Tasl
  6. Error Start Event
  7. Cancel End Event (NPE, probably eclipse bug)
  8. Multi instance (Both Parallel or sequential)

Supported BPMN Elements:

  1. Start Events:
    • None Start Event
    • Timer Start Event
    • Message Start Event
    • Signal Start Event
  2. End Events:
    • None End Event
    • Error Event
    • Message End Event
    • Signal End Event
  3. Boundary Events:
    • Message boundary event
    • Error Boundary event
    • Signal Boundary event
    • Compensate Boundary event
    • Intemediate events::
    • Message boundary event
  4. Intemediate events:
    • Intermediate Message throwing event
    • Intermediate Message catching event
    • Intermediate Signal Catching event
    • Intermediate Signal Throwing event
    • Timer intermediate event
    • Signal throwing event
  5. Sequences:
    • Sequence Flow
    • Conditional Sequence Flows
  6. Gateways:
    • Exclusive gateway
    • Parallel gateway
    • Inclusive Gateway
    • Event based Gateway
  7. Tasks:
    • User Task
    • Script Task
    • Service Task
    • Receive Task
    • Manual task

Update in Web Service Mock demo

In follow up of my previous post about mocking web services, I rewrote the sample demo to better show the concept.
Not the there is a class multiplier that uses adder service. Now it is clearer to see that Multiplier is the class we want to test and not the adder.
The adder is mocked only to test the multiplier. I also removed the traditional way of creating a concrete web service for testing. It may confuse the reader.
The updated source is available in github: https://github.com/smirzai/webServiceMockDemo

There is also a available.

Dynamic Tasks In Activiti

Requirements:

Here is a hypothetical requirement:
We are talking about a help desk system here. When an incident occurs, a human task is created and assigned to the help desk supervisor. Supervisor may fix it himself and complete the task. As of now, this is standard activiti human task.
Now the supervisor may need to ask the idea of network admin. And/or he may need to ask oracle dba to do something. He needs to add unlimitted number of tasks, assigned to various actors.
Now it is more complex, but again could be implemented with SubTask undocumented feature of actviti.
In the next level, in some cases the supervisor may need to add two  or more consecutive task instead of one single task. Suppose that he wants to get the idea of Oracle DBA and forward it automatically to the software developer. Each of  the following actors, may in turn do the same thing which the supervisor can do. They in turn, may add one, two or more step task series to the process.
The process shall continue, when all the created tasks are completed.

Here is where it gets interesting.

Solution interface:
Here is what I came to, to address requirements.
Let’s start from the high level diagram.

dynamicTaskDiagram
After the start of the process, Service task creates the first dynamic task. Java delegate class simply calls  function from DynaTaskService service:

public class DynamicTaskDelegate implements JavaDelegate {
   @Override
   public void execute(DelegateExecution execution) throws Exception {
      MyUnitTest.dynaTaskService.createOneTask((ExecutionEntity) execution);
   }
}

This will create a dynamic with special behavior. Actually in the DynaTaskService two methods avalable. One for creating one single dynamic task, the other one for two consecutive dynamic tasks:

public interface DynaTaskService  {
    public void createOneTask(ExecutionEntity execution);
    public void createTwoTasks(ExecutionEntity execution);
}

calling createOneTask will create an instance of the unattached single User task, you can see in the diagram. Similarly the createTwoTasks function, creates an instance of unattached sequence of two consecutive tasks.
The next actors, can also call the same methods to create as many tasks as required.
Creating each task causes a counter process variable to be increased.

The magic happens when a task is completed using taskService.completeTask if the task is a dynamic one and there is some other task following, it will be executed as usual. But if the task is dynamic and it is the end task, then another counter, counting number of completed tasks will be increased, line of execution will be terminated, and the receive task is signalled. The gateway after receive task, checkes if the number of completed tasks are equal to number of created tasks. If there are still open tasks, then the loop continues.

This fully addresses the strange non usual requested functionality.

But let’s see how it works under the hood:

How it works:
Tasks are created by calling createOneTask or createTwoTask.
When called createt task function does these steps:

  • increase the counter for created tasks, create if it does not already exist
  • creates seperate execution for the new task
  • create a new task regarding a specific naming convention, with the proper activity name already available in Diagram
  • relates the created task to the created execution

user can call the create task functions arbitarily. The normal activity behavior for these kind of dynamically created task is simply to remove them, and remove the execution if nothing is coming afterwards. This is not exactly what we want.
To change this behavior, we need to override default activiti behavior for completing tasks. Here MyCustomActivitiBehaviorFactory factory comes into the play. It creates our custom behavior MyUserTaskActiviyBehavior when asked by engine.
The most important part of logic is handled in MyUserTaskActivityBehavior. Here is a list to show what is done in the behavior in short:

  • Using name of the task, determine if it is a dynatask. It is not used right now, but may become handy.
  • if there is an outgoing sequence, just follow it normally
  • if there is no outgoing sequence, it means the task is at the end of execution. increase completed tasks counter, end the signal the recieve task on the parent execution

The source file is available in github

JMX feature added to Activiti

I just contributed the JMX support feature to Activiti. It is merged and will be available in the coming release (release 5.16.4) of Activiti.

It is specially useful for people in operation teams. It reveals performance information and enables the operator to change some parameters on the fly.

To enable the feature it is enough to add the activiti-jmx module to dependencies. Jmx Mbean server will be automatically found and configured with the default settins.

Default configuration makes the JVM instance visible in Jconsole. If it is not visible for any reason, it can be found using this url: service:jmx:rmi:///jndi/rmi://localhost:1099/jmxrmi/activiti using jconsole. By default Mbeans are exposed in org.activiti.jmx.Mbeans path.

As of now these informations and operations are available via JMX:

Attributes:

  • getting process definitions
  • getting deployment

Operations:

  • Undeploy
  • suspend, activate process definitions by id and key

Default configuration can be changed by putting these parameters in config file or setting them directly via API:

  • registryPort (Default is 1099)
  • mbeanDomain (default is “DefaultDomain”)
  • connectorPort
  • serviceUrlPath (Default is /jmxrmi/activiti)
  • createConnector (default is true)
  • disables (default is false)
  • domain (default is org.activiti.jmx.mbeanObjectDomainName)

New Mbeans are easy to add thanks to created JMX annotation. You need only to annotate attributes with @ManagedAttribute and methods with @ManagedOperation. The only possible annotation parameter is “description”. The given description will be shown as Mbean description.

There are test cases, which bring up the jmx server and actually query and change the information via JMX. They can be used as the sample for have a taste of what a jmx client can look like.
DeploymentsJMXClientTest and JobExecutorJMXClientTest are such test cases.