Category Archives: Uncategorized

Installing kuberenets on vagrant

Just to get some knowledge and test products on it, it is vey useful to be have access to a cluster. One easy way is using Vagrant. Here is how I did it:

I am using a bare metal server running Centos 7.4.1708

  1. Software installation
    • Vagrant 2.0.3
    • VirtualBox 5.1.34r121010
  2. Setup vagrant
    1. Copy vagrant file from https://gist.github.com/smirzai/8724cb36bd7fcbe4d289b86586b70c5a
    2. vagrant up. You will have four machines named n1, n2, n3, n4. n1 will be the master node. Each machine will have two  network interfaces. Here the local is 192.168.0.0/16
  3. for each node prepare the machine:
    1. sudo apt-get update && sudo apt-get upgrade
    2. Install docker, kubectl, kubeadm, kubelet as described in installing kubeadm document
    3. swap off -a (This is not persistent. Should be done every time a machine restarts)
    4. The following changes should be enforced in /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
      1. Environment=”KUBELET_EXTRA_ARGS=–node-ip=192.168.77.10 –enable-debugging-handlers=true  –cluster-dns=10.244.0.10 –cgroup-driver=cgroupfs –cluster-dns=10.244.0.10″
      2. The above –node-ip should be replaced with ip of the machine
      3. $KUBELET_CGROUP_ARGS should also be added to the line starting with ExecStart=/usr/bin/kubelet
      4. restart kubeadm by executing:
        1. systemctl daemon-reload
        2. systemctl restart kubelet
    5. Initialize the cluster:
      1. sudo kubeadm init –apiserver-advertise-address 192.168.77.10 –pod-network-cidr=10.244.0.0/16 –service-cidr=10.244.0.0/16
      2. The init command prints the statement for joining the nodes. Write it down somewhere.
      3. Update the kubectl config file:
        1. mkdir -p $HOME/.kube
          sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
          sudo chown $(id -u):$(id -g) $HOME/.kube/config
        2. or execute the following as roo:
          export KUBECONFIG=/etc/kubernetes/admin.conf
      4. Install flannel:
        1. download flannel creation yams file:
          wget https://raw.githubusercontent.com/coreos/flannel/v0.9.1/Documentation/kube-flannel.yml
        2. add “–iface=eth1” to the  flannel.yaml  in the following line:
          command: [ “/opt/bin/flanneld”, “–ip-masq”, “–kube-subnet-mgr” , “–iface=eth1”]
        3. create flannel cid network: kubectl -f flannel.yaml
        4. Now all nodes should be ready
        5. Join the n2, n3, n4 by executing the join command written down above in init step

 

Fast daily report bash script

I prefer to create a small asciidoc file as my daily report with minimum information about what I have done and some small notes and labels on what I have done.

Several times a day, I have to revert back to daily report and update it. To speed it up, I created this naive shell script.

It simply checks if the daily report file exists for today. If it does not exists, it copy one template, updates the date to today and return the full file in any case.

using this alias

alias edr=”emacs `~/opt/dr`”

I can create or update the daily report with 4 keystrokes.

Activiti docker image

Docker is a fantastic virtualization tool for running different images very fast and with minimum effort. There is a hub available with many pre-created ready to use images.

With docker it is matter of minutes to download and run images for Activiti explorer and rest Api.

It would be greatly helpful for people who want to play and evaluate the tool. Also for people who make demos and present Activiti.

First you need to install docker, if it is not already done. Installation is very straight forward. Binaries are available for nearly every major OS.

After installation, you need to pull the image:

docker pull smirzai/activiti-single-image

Downloading image may take some time, depending on your internet connection. After that you can easily run explorer or Rest-API.

Executing this command runs the explorer in the container:

sudo docker run -p 8080:8080  -t -i smirzai/activiti-single-image:latest explorer

After less than a minute, container will be running. You can access the explore using such an URL in your explorer:

http://localhost:8080/activiti-explorer

voilà ! you are there.

Similarly you can run Rest API, using such a command:

sudo docker run -p 8080:8080  -t -i smirzai/activiti-single-image:latest rest

Explorer is the default, if you omit the last parameter.

We are working on making it official out of my personal docker hub.

 

 

problems of confluence as document writing tool

Here is the list of problems I faced in using confluence for writing documents and then converting them to word or PDF.

1. Master/Slave versioning:

if you have a master document which consists of several child pages, each of them will have their own version history. consequently you cannot use versioning that comes with confluence for parent/child pages, unless you are happy with having several change history tables, with different version numbers in the final document.

2. Offline usage
You cannot work on your documents where you are on a train.

3. Loosing content
Ok, this is only one time experience. After half an hour working on documentation, I got a horrible error message “you have enough credentials” and all my work vanished.

4. source control
you cannot push your documents in source control. This means that you have to keep the source and the documentation in sync manually.

5. fine grained versioning
I don’t really want to increase the version each time I change a comma in the text. As far as I know there is no tool, to let me decide when version should be incremented.

6. hard editing
If I find a typo in my docs, which I often do, and I want to correct it, I need to scroll up to the top of the page, press edit button (this can be done faster by keyboard shortcut), scroll back and find the typo and correct it, then I need to scroll down to the bottom of the page and press save (For the last part I think ctrl-s is a keyboard shortcut). This can be cumbersome in long documents.

Exporting diagrams from Bizagi to Activiti Designer

Activiti comes with two modelers, eclipse based and web based. Activiti does not promise NO-CODE workflow and modelers are also designed having a developer as user in mind, specially the eclipse one. One can easily forget about eclipse based designer when trying to use it to discuss a business process with a domain expert or a business analyst. They need some more graphical tool than eclipse.

Bizagi is a robust clean free tool for such purposes. One can use it easily to draw a conceptual BPMN diagram and even simulate it in some extend.

But what if you decide to execute the model using Activiti ? In older versions, it was not possible to reuse the diagram available in Bizagi. The whole process diagram needed to be recreated in activiti from scratch, blank paper style. The most recent version (I am using version 2.5 for this test), supports exporting BPMN diagram to various formats including BPMN. This format is what activiti uses by default. So theoritically it should be possible to open the diagram created in Bizagi  inside eclipse based designer and edit it there.

But they do not funtion always as advertised. I did a short test to check how possible it is.

First of all, it is important to notice the two tools serve more or less different purposes. Activiti designer cover all the detailed required information to really execute the model by its own engine. Diagram in Bizagi is more a conceptual diagram and lack those details. So obviously in best case, the converted diagram is not executable. Only the base structure will be converted and all the details, class named, error codes, and so on should be added manually. Not ideal, but way much better that redrawing every thing from scratch.

Exported diagram can be opened in Eclipse designer and usually the basic patter nis readily visible. It is fine as long as you have not tried to edit something. The smallest change and save makes the whole disagram be unreadable the next time. The reason is that for any reason activiti does not understand the Collaboration semantics produced by Bizagi. Whose fault is it, should be checked and fixed. Activiti designer only uses it for graphical representation after all, no effect on the execution.
So one workaround is to delete the whole content of <Collaboration> tag as first step before trying to save the diagram.

The above trick makes the diagram editable in eclipse designer. Most of the commonly used BPMN elements are compatible from Bizagi to Activiti, but not all of them. I did a quick test on what is supported and what is not and here is the result:

UnSupported BPMN Elements:

  1. Subprocess
  2. CallActiviy
  3. Shell Task
  4. Camel Task
  5. Mail Tasl
  6. Error Start Event
  7. Cancel End Event (NPE, probably eclipse bug)
  8. Multi instance (Both Parallel or sequential)

Supported BPMN Elements:

  1. Start Events:
    • None Start Event
    • Timer Start Event
    • Message Start Event
    • Signal Start Event
  2. End Events:
    • None End Event
    • Error Event
    • Message End Event
    • Signal End Event
  3. Boundary Events:
    • Message boundary event
    • Error Boundary event
    • Signal Boundary event
    • Compensate Boundary event
    • Intemediate events::
    • Message boundary event
  4. Intemediate events:
    • Intermediate Message throwing event
    • Intermediate Message catching event
    • Intermediate Signal Catching event
    • Intermediate Signal Throwing event
    • Timer intermediate event
    • Signal throwing event
  5. Sequences:
    • Sequence Flow
    • Conditional Sequence Flows
  6. Gateways:
    • Exclusive gateway
    • Parallel gateway
    • Inclusive Gateway
    • Event based Gateway
  7. Tasks:
    • User Task
    • Script Task
    • Service Task
    • Receive Task
    • Manual task

Update in Web Service Mock demo

In follow up of my previous post about mocking web services, I rewrote the sample demo to better show the concept.
Not the there is a class multiplier that uses adder service. Now it is clearer to see that Multiplier is the class we want to test and not the adder.
The adder is mocked only to test the multiplier. I also removed the traditional way of creating a concrete web service for testing. It may confuse the reader.
The updated source is available in github: https://github.com/smirzai/webServiceMockDemo

There is also a available.

Updating git from svn

I just had to improt a huge svn repository into git.
If you spend enough time on importing, importing with git svn clone is not that hard. It keeps the history, and works very good.
The problem arose when I tried to remove big jar files from the repository. I planned to move it to maven and it was not a good idea to keep jar files in repository.
Simple removing them, left the footsteps in the history and made it unncessarily big.
I tried to rewrite history the way explained here. It took more than one week. I cancelled it.
This tools helped me:
http://rtyley.github.io/bfg-repo-cleaner/

It is rally fast and efficient. It did the job in matter of hours.

I cleane it up and pushed it into repository. Afte one week I found out, the team has done some commits on old svn.
using svn git, meant to import every thing from scratch.
I decided to write a python script, create one patch for each svn revision. Then applying them and commiting with the same description as subversion. It needed some manual work but was much faster than importing the whole svn

How to mock web services

In black box testing, sometimes we need to test a system which consumes and produces web services. Here is my experience on how to test such a system.

you need to call a the exposed web service and check how the consumed webs services are called. Exposed web service is already provided by the system, and you need only a client to call it. I used CXF. very straight forward and easy.

But the consumed web services are not there yet. They are not in the scope of tests and mostly not in our control at all.

One approach is to create fix implementation of thos web services, again using a library like CXF and  when the callback functions are called, check that the parametrs are passed correctly and resturn a suitable result for test of other parts at the same time.

This approach works fine only for the simplest cases. If you want to test different scenarios, the fixed mock web service should react differently and you have to put the logic inside it. Sometimes you meay need to have more than one implementation and re-publish the service with the alternative implementations. Not beautiful I admit.

A more beautiful approach is to use mocking frameworks. Here I have selected Mockito.

Here are the steps:

First you have to create a mock object, a normal one based on the service interface. I do it via annotation. like this:

@Mock
static ServiceInterface serviceMock;

But this should be called from a web service. For any strange reason, it could not be done directly. I had to create a reflecter class to do the job:

@Mock
public class ServiceProxy implements java.lang.reflect.InvocationHandler {
    private Object obj;

    public static Object newInstance(Object obj) {
        return java.lang.reflect.Proxy.newProxyInstance(
            obj.getClass().getClassLoader(),
            obj.getClass().getInterfaces(),
            new ServiceProxy(obj));
    }


    private ServiceProxy(Object obj) {
        this.obj = obj;
    }

    @Override
    public Object invoke(Object proxy, Method m, Object[] args)
        throws Throwable {
        Object result;
        try {
    
            result = m.invoke(obj, args);
        } catch (Exception e) {
            throw new RuntimeException("unexpected invocation exception: " +
            e.getMessage());
        }

        return result;
    }
}

then it is easy, you can define your web service mock and start it.


service = (ServiceInterface)         ServiceProxy.newInstance(serviceMock);
serviceEndpoint = Endpoint.publish(
				"http://localhost:8080/myService", service);
 

Every thing is ready now. you can run the system and use the mock, just like a normal mock.
As an example of the usage, this code tests if a method is called in web service. Obviously all the mock language of Mockito can be used for more complex tests.

verify(serviceMock).myMethod(anyInt);

 
Enjoy your testing !

To present it in a more concrete way I have created a sample usage. It could be found here: https://github.com/smirzai/webServiceMock.git

There is also a worldline version. It uses resource locator to find web service.

Synchronous and Asancronous binary tree comparison

if you have not taken “Principles of reactive programming” course about advanced Scala and Aker framework, probably you are not using internet properly, think again. It is a wonderful course from EPFL, one of the hardest in Coursera.

To pass this course you have to complete some programming excersized. One of them is “Actor Binary Tree”, In this example you have to implement an asynchronous search binary tree. When you insert a node, it is distributed asynchronously through the tree. Nothing is locked and this makes it really fast. The sender actor gets acknowledged when the insertion is done.

In theory the idea is really beautiful,  but how useful is it in reality? Why we have to use this relatively complex architecture. The answer is that because it is asynchrouns nothing is locked and you should have a very high performance.

But how it comares with a similar pure java implementation ? For consistency usually we lock the tree. That definitely will not compete. But if we use an immutable version we do not need to worry about the consistency.

To do the comparison I searched and found an immutable binary tree in Kansas State University.

To compare I will insert large numbers of nodes in both implementations and wait untill all are acknowledged and measure the time. In Java implementation different number of threads is tried.

BinarySearchComparison

 

The maximum TPS of Scala version is 2952 and java version 2701739, more than 90 times.

So the conclusion is, at least if the operation itself is fast and not IO intensive, it is no use in usage of asynchronous type of programming. The overhead of aker is considerable if the operations themselves are fast.

 

The source for java implementation is available in github.