Designing and coding complicated Tasks in workflow based systems

it's almost my second year participating in a project that features a J2EE application that uses a work-flow engine and the related BPM tools. Our main goal is to map our client's manual business processes to fully or semi automated work flow process that usually involve a large amount of human interaction through tasks (aka workitems).

In this period I was lucky enough to participate int the develpment using 2 totally different technologies. First attempt was using BEA's (long gone now) WLI technology and the second most recent and I hope successful is using JBoss JBPM technology.

I could write long paragraphs ranting about workflow engines, BPM design tools and the overall state of this specific technology frontier. Eventually at the time being I can give one big advice.

Never .never try to implement something like this using a technology that was dictated by some funcy clever sales person. It is most probable that you will end up with a disaster! Don't let your's or your client's sales team (if you can) pick the technology for your, especially if the selected package is proprietary. You are doomed to fail!

Anyway more advices on choosing a good work-flow engine and of course when you really need such a tool, maybe in a future post.

So the usual problem is the following. You client has a lot of manual business processes and wants to map them using BPM using a reliable work-flow engine. So you think, this is a good scenario to use BPM, you go for it.

In case you are not familiar with such technologies, you start evaluating some of them. In most cases, all the demos work, all the demos try to prove that my engine does everything for you, my engine will make your developers happy, my engine will require almost drag n drop development. In some cases clever salesmen convince even your client saying At some point you could even draw your business processes on your own - no technical assistance would be required'. The client is now smiling they want it NOW!

Well .well .in the real world, at least in this point of time all the above are .just wishful thinking.

One of the most important part of the demos for a work-flow engine, is the task management / creation. Al the demos have several wizards that using the engine's API, create Tasks and the equivalent simple demo pages. Some people try to convince you that since you have so much human interaction in your processes you will most provably want to have a tool that is going to create and wire tasks for you, create pages that are going to map your business data, save them, retrieve them and coordinate their assignment through the engine's internals.

Unfortunately all the above are true but only in the context of your demo. In the real world, in a real complex application your client's task management and the actual tasks are going to be very complex pages, loaded with features, data and complicated wiring with the business process. The task designer will most probably prove to be not sufficient to implement such complex scenarios and then this is where your nightmare begins as a developer.

In case a decent API is provided you will try to use your engine's facilities I to by pass the default page generation, to wire more features to the default tasks. There is a point where you will really want to do very custom things (and this is a very realistic scenario- in every application).

You just stand there, watch your poor task creation API that can not be extended or even hacked and you think what should I do?

Most probably that you will end up with a hybrid model. You will still use the task API facilities provided by your engine and mix them with your own custom constructs (new tables in your DB model). You will use soft or hard foreign keys trying to build complex pages and functionality linking with the workflow tasks. It will work for sure, after lots of testing and development and effort.At some point you will think wait a minute - when we first saw this technology it was promising it will provide everything to us, what am I doing now? Am I extending the engine's model, why?'

Valid question. In fact my short experience proved that you can avoid all this double effort, this wiring of different technologies, making your system highly coupled and hard to maintain.

So what is the solution. The answer is twofold

a) use an open source workflow engine (it is obvious why, you control the source you control it's features or even if it is dictated by some other parties involved to use a proprietary one - make sure that you have enough access to it's internals through the API and appropriate constructs are provided. In other words, try to avoid making your workflow engine your application - it is a part so code carefully and avoid dependencies!

b) When it comes to designing and coding tasks, do it on your own. In our case when we switched 100% on coding tasks with our own constructs meaning not using the engine's out of the box objects or constructs we could see that our productivity boosted. The web layer development for the tasks was much easier, the wiring of the tasks with the engine became more flexible and of course we had removed another dependency from our application. It was our code, it could be extended and maintained by us.

Even when we switched to JBPM which is by far one of the most simple yet powerful workflow engines available we decided not to use the Task Node provided by JBPM. Design wise we picked a very simplistic solution. Add a node to create your 'custom Task Object_ and then put a wait state in order to simulate pause until it is completed feature. In our cases all our tasks were falling into this example.

Your custom task functionality will of course have mechanisms that are going to control things like - retrieval, saving, archiving and even date related functionality like (valid until XX date etc). It is not that difficult or how can i say a lot easier if you need to maintain 2 different models in order to implement the same thing!

Maybe all the above sound extremely simplistic but IMHO in the BPM development world if you want to see your system running and scaling you have to keep it simple.