Project management, queuing, avoiding bottlenecks

 So you have many tasks to complete, limited time, and you accept that not everything will be done.

The problem is to fit as many problems into the time as possible, and exclude the few "big" problems.

Turing already proved that you cannot know how long a problem will take before actually completing it.

The problem is that a problem that looks simple with available information expands under solving as we discover more.

The simplest solution is to allocate time, and just interrupt problems when timeout. Go to the next problem in the list and keep cycling the list. Multiple passes is the absolute best strategy in exams. Complete the easy questions first and claim those marks. work through the paper in multiple passes so that you are left with the questions you find hard. This way you slowly reduce the rate of mark gaining.

This blog has already analysed rates and shown that the best way to completion in exercise is to maintain rate. I spoke to a competition swimmer recently and he agreed although said that first and last length were not so controlled and obviously in last length you give everything. But in exams and other non linear problem solving situations where we don't need to do everything, we want to leave the hardest to last and set our rate on the easy stuff. In fact this would make a good optimisation study. #TODO.

The problem with interrupts is that you might end up stacking a problem that is almost complete. Stacking in computers and real life is an expensive business. Re-establishing context takes time. You put a problem down and come back in a week and your short term memory is completely cleared. You need read it all over, think through again and stimulate long term memory into the short term context to get the problem running again.

Daniel Dennett points out that its no coincidence that computer design fits with human mind so well. We based the computer on our own thinking. Caching, short/long term memory, context, interrupts are all features of our own daily mental space.

So a semantic component to scheduling is also useful not just blind interrupts.

The biggest waste of time is when something does not work and we go on a search to fix it. This would be like going looking up a shop, going to the high street and finding it's not there. The obvious next step is to start searching. Perhaps you got the address wrong, perhaps its closed. You start asking people. This now enters an unknown period and the time is ticking.

As a rule of thumb you spend another 2x the time you allocated. So if you had 30mins allocated to go the shops for something, you now allocated 1h30. The reason is that simply quitting and going home to the next task wastes all the time spending gaining context. You are now in the context of the high-street. That is expensive and you need use that while you can. I took the 2x from a probability study of marriage. You sample for 1/3 the time and find the environmental "best" you then spend 2/3 the time searching for anything better than this "best." But in questions of waiting this seems about right.

Consider the bus has not arrived problem. How long do you wait until looking for another way home? Well if you have waited 10 minutes before thinking "now there is a problem." Wait another 20 before doing anything about it. That 10 minutes contains information about how much of a hurry you are in, and it stops us wasting the "context." We are already waiting at the bus stop, if we rush off now we will miss anything that does arrive. This is a slightly unusual problem as the chance of the bus arriving increases as time goes on given that the company is trying to maintain a constant rate. 2 buses will come soon. However at what point does probability increase change to decrease as we realise there is serious problem and perhaps the road is closed or the bus route cancelled. This has been a long term problem that still not solved. #TODO

The method that is working for me right now is lists.

1) With available information you write out a schedule. Not just a casual one an actual list of instructions. It is cheap to write instructions, it is expensive to actually do them. So by writing out a detailed list we get to review all the available information and see problems cheaply before they arrive.

2) We sort these lists into what look like the easiest, maximum impact problems. We want to maintain the maximum "rate of target achievement" like in the exam strategy. 

3) We should weight the problems by whether they are actually important. In software development an easy problem that simply refactors code is not going to be seen by the user. It may have long term benefits for maintainers as the code is easier to read and adopts more universally identifiable patterns, but for maximum impact of the project this problem goes down the list anyway.

4) Now we actually start at instruction 1 of the first problem and work through. Now inevitably information will be thrown up. This is either an error in the scheduling process (we thought about something wrong) or something unexpected (new information).

Obviously the actual scheduling process is a problem itself. We find time for that, and like all problems it itself is governed by the same issues. So we may even make mistakes in the scheduling process that we must handle.

5) Which ever way a simple instruction ceases to be a simple tick off step. Now this is where we step into Review and prepare to Stack. We note what the problem is and what we think we need to do as a revision to the schedule.

6) We review this. Does the new plan change the order. Is this problem still important enough to pursue or is something else a better candidate now?

If we can see a better candidate we do the expensive context stack, recording the new instructions for how to proceed when we pick it up again and inserting it into the existing list.

7) Now we start the next problem.

So this way we avoid the performance killer of chasing a rabbit down the rabbit hole. If we are not careful half a day can easily be wasted chasing a difficult problem that keeps twisting and turning.

It also means that we do not enter that "reflex state" of mind where we are just blinding "pursuing impulses." Its lovely to solve problems in a "shoot'em up" mentality where we just do the next thing that's on our mind. We can get into an actual fight with a problem, exploring this and that very dynamically. But while this is fun it often leads us into making big code changes to what turns out to be a dead end. We realise something and end up backing out of maybe an hours code.

The problem is that Thinking Is Fast, Doing is Slow. In the football film "Goal" there is a great scene where the trainer asks the asthmatic footballer to run after a ball her has kicked. "What do you learn?" "It is quicker to pass than to run" is the answer. Likewise it is much faster to think than to to do. This is WHY we ever evolved thinking in the first place.

Now obviously thinking is its own rabbit hole and we can get stuck in endless planning and reviews. But it should ALWAYS be the first step.

So when we face unexpected things do not instantly follow the obvious road and get led off in a wild goose chase without returning to the Project Plan and Review. Review is the key thing to keep on track. An overview and some thought about what to do with new information is essential to keeping target rates up and maximising the rate of delivery.

Okay blogging is not very good constructive in target delivery and need actually implement the above right now.