Eclipse tweaking

My eclipse died today (again)
I did the mistake of installed a bunch of plugins and then a few more, and lo and behold, the eclipse slowed down and a day later failed to start.
Eclipse plugins/distributions hell makes "DLL-Hell" seam a likes a pleasant place.

I just installed a new clean replacement. Few performance tweaks worth doing.

• Change the ridiculously low memory cap of -Xmx384m to at least -Xmx1024m in eclipse.ini 
• Use a shared, external plugin folders, so that you will not have to re-install all of them when you decide to upgrade to the next great eclipse version.  Just add this to the eclipse.ini Dorg.eclipse.equinox.p2.reconciler.dropins.directory=/path/to/shared/plugins/directory.
• Install this tiny but great plugin: http://code.google.com/p/tarlog-plugins/ which allows you to open shell/window and to re-size with ctrl+.  Just put it in the plugin directory above.
• If you have use an SSD drive as the eclipse drive. The change in start-up time is incredible.

C# ain`t too bad

C# ain`t too bad

Nothing is really free in Microsoft`s world , but you can start by using the express editions of Visual C# 2010 and Visual Web Developer 2010 .
It amazing how much better the (java) free eclipse IDE is compared to the free visual C# , but if you are willing to put few hundred $`s , The Ultimate version+Resharper should give eclipse a very good competition.

Few notes:
LINQ is absolutely brilliant.

Delegates/Events are cool too. They are explained here.

WPF DependencyProperty is explained here.

NUnit is a great testing tool , especially with the new attributes and constraints
But it does not integrate to VisualStudio Express (and Microsoft does not allow plugin/extensions to the epress edition)
Don`t worry , just do:
1. Add reference to NUnit in the project. (Add reference -> .Net -> component name=NUnit frameworl
2. Open the nunit-x86.exe (on windows7) , create a project and use "Add assembly" to target the relevant project dll/exe.
3. Don`t run it manually, instead change the Program.cs to run it, instructions here . Note that you will need to add the nunit-gui-runner.dll from the NUnit ../bin/../lib directory

The Agile Dogma 

Dogma :: "prescribed doctrine proclaimed as unquestionably true by aparticular group"

In the recent years I worked both under the waterfall-dogma and under the Agile-dogma.

I have come to three conclusion:

1. Both Dogmas are wrong as a dogma.
2. Both have valid use-case for particular projects.
3. Most developers don`t understand the difference between prototyping methodology and product-development methodology. Agile is the second, not the first.

When you start a new software project, you need to decide which product-prototyping method is right for you and then which software-development dogma is the right for the project. Choosing the wrong one is a clear path to failure, so choose wisely.

Step 1: You think you have a great product-idea. How can you know it`s indeed a great one.

First ask yourself, is my idea is a bright new product which was never commercially tried before (GPS-enabled Sun-glasses) or is it only a small improvement on an existing successful product
( new email client )

• If it`s a new product - you must create a minimal prototype to test if it will actual sell.  It should be very fast - up to a month`s work.  Agile/Waterfall do not apply for such short time frame. 
• If it`s an incarnation of an exiting product - great! someone else already "prototyped it" for you. Fill free to use their product to and jump to the next stage.

Step 2:  You verified the idea and shown a prototype which people love.  Should you have one public-release or a lot of small public-releases?

1. What is the use-pattern of your product?Is it a "use-once"(a movie) , "use-for-few-weeks"(most games) or "continues-yearly-use" (most websites) ?  
2. Can an existing user get the new version of  the product very easily?
   Is it automatic(a website), a manual-update (mobile-app) or plain-impossible(firmware,hardware)

If the answer is a clear "YES" to both of the questions above, go for multiple-releases.

If the answer is no even to one,  you need one big release (didn`t talk about agile/waterfall yet)

Let`s see some samples:

• For a web-site like an online-store/news-site the answer for both is a clearest yes. The users use your site on a continues-yearly-use and each time a user enters the site he will get the new version automatically.
• For a media (movie/presentation) the answer is a clear no, as most of the users will see the movie once in the first month and will not bother to re-view it once you release a new version.
• For downloadable game, the answer is a clear no. Although most game have an auto-update feature for bug-fixes, most users will play the game only in the first month. 
• For firmware for hardware-devices the answer is usually no. Although the user use in continually  99% of the users won`t upgrade the firmware version.
• For a software library (like logging framework) the answer is again a clear no.  Other developers will not update your library automatically, and will usually do it once  a year, when they need an extra feature the new version has.

If you answered no to one of the above, you need to have one ultimate public-release
But, on some products , you can still have multiple "closed-beta releases".

1. First find a large pool of  "beta" users which you can ship your product too. They should be the typical audience , but their size should be up to 1% of the intended users. (You can pay them)
2. Divide them to multiple distinct groups.  If you plan around 5 iterations, create 5-8 groups.
3. In the end of each iteration, use one "fresh" group and get it`s feedback. You will not reuse that group again
4. In the last iteration , release to the general public.

The large pool of users is a must. Using the same person all the time, usually your product-manager or in small-startup`s your spouse, gives very bad results.

Sometimes you can`t find any users which will test your product. In this case you can still create internal-releases or internal milestones in which each feature is finished and ready to be internally tested. This is true to both agile and waterfall approaches.

The obvious Bottom line is "NEVER DO THE FIRST-TEST JUST BEFORE PUBLIC RELEASE".  You will be surprised how many companies do just that.

Step 3:  You know how many releases you can do.  What software principle should you use?

All software products can be separated into multiple small independent features. This is true if you are building a algorithmic-focus search-engine , or a design-focus web-site.

So first divide it to these features and work on them.

Waterfall is great if it is done on one small feature. Meaning you will spend a 1 day on design, 3 days on development&unit-testing ,and then 1 days on system-testing.

But what of the features that take few months to develop?

Well, first try again to break them into sub-features, most features can be broken with enough thought.

If you really can`t , and it`s rare and usually limited to algorithm-intensive or framework-intensive work, then go waterfall on this particular feature.  Let me emphasize that these kind of features are very-rare in web/media/mobile products and most developers today should do it very rarely.

So bottom line here, design+implement+test in the waterfall way is good as long as it is for small features. (some will say this actual == most-of-agile)

Heroku and Amazon ElasticBeanstalk

Amazon Elastic Beanstalk is a platform-as-a-service solution for web-application development.
What it means is that you upload a java WAR ("jar for the web) file and Amazon takes care of hosting it on a pre-installed Tomcat web server.

Heroku was created by a different company to sugar-coat the Amazon platform.  For some extra money per month, you will get:

• much much better documentation
• better, simpler, eclipse-plugins
• Easy configuration of external services like non-sql DB and Caching.

If you are working on a small scale project , and your time is precious , go Heroku.

Principles

On requirements

1. You ain`t gonna need it - Always implement things when you actually need them, never when you just foresee that you need them.

2. MoSCoW method (Must Should Could Won`t)All requirements are important, but they are prioritized to deliver the greatest and most immediate business benefits early. Developers will initially try to deliver all the M, S and C requirements but the S and C requirements will be the first to go if the delivery timescale looks threatened.

On project scheduling

1. Brooks`s law - adding manpower to a late software project makes it later. "Nine women can't make a baby in one month"

2. Parkinson`s Law - always time each task. Work expands so as to fill the time available for its completion

On Design

1. Analysis paralysis - over-analyzing (or over-thinking) a situation, so that a decision or action is never taken, in effect paralyzing the outcome. A decision can be treated as over-complicated, with too many detailed options, so that a choice is never made, rather than try something and change if a major problem arises.

2. KISS principle - Keep it simple stupid. The principle is best exemplified by the story of Johnson handing a team of design engineers a handful of tools, with the challenge that the jet aircraft they were designing must be repairable by an average mechanic in the field under combat conditions with only these tools. Hence, the 'stupid' refers to the relationship between the way things break and the sophistication available to fix them.

.... It seems that perfection is reached not when there is nothing left to add, but when there is nothing left to take away

3. Rule of three ( not two as many think , three!) - Rule of three is a code refactoring rule of thumb to decide when a replicated piece of code should be replaced by a new procedure. It states that you are allowed to copy and paste the code once, but that when the same code is replicated three times, it should be extracted into a new procedure.

scala

AKKA http://days2010.scala-lang.org/node/138/162

Eclipse plugin for Eclise 3.4 is incredibility bad.
Eclipse plugin for Eclipse 3.5.2 is better.

tutorials
http://www.scala-lang.org/docu/files/ScalaTutorial.pdf
http://www.scala-lang.org/docu/files/ScalaByExample.pdf

Postgres 9.0 hot-standby

Postgres 9.0 have a new feature "streaming-replication" + "warm-standby". In few words , it allow asynchronic replication between master and slave (there is a delay , but it is a small one) and the slave can do read-only queries.

synchronios replication is only planned for 9.1

It`s a good time to recap the different sharding/master-slave options and their roles.

At first you develop the code for one DB instance and you assume three things about it:

1. It will never fail - the machine,process and disks are eternal.

2. data-scale: It can grow is size as much as you want - terra-bytes of disks space.
3. query(users) scale : The machine is as fast as you want - It can supports millions of fast queries in parallel.

After few months , you deploy it and then find out that these 3 things are not so easy to achieve. Lets see what are the (Postgres) solutions for them.

query-scale

1. Scale-Up : Still one machine, but buy the best one money can get - 8 cores,32GB are relatively cheap. But twice as strong machine will cost a lot more the twice the money. If you need x2 or x4 , switch to scale-out.
2. Scale-Out :
   2.1 Scale reads not writes: (good for mainly read DB)
   Use one master server (read&write) and multiple read-only slave servers which get periodically updated from the master server. The master and slave share the same data.
   Postgres 9.0 supports it out-of-the-box
   2.2 Scale writes by sharding

Fault-tolerance

For transaction-DB where no record can be lost , this is a hard task. For data-warehouse which may loose the latest few minutes of updates , this is much simpler.

No data loss:

1. Shared-disks : the master writes to a shared storage. When it fails the slave mount it and loads the data. It is a "code-standby" as it can takes few minutes to recover.
2. pgpool II - slows the updates on the master!
3. synchronios log shipping (should be in postgres 9.1) - does not (considerably) slow the master. Failover take 15 seconds (10 to detect failure , 5 to start the warm-standby)

Some data loss allowed:

1. Postgres 9.0 warm-standy & streaming-replication
2. Slony-1 updates the slave using triggers

Data scale

Start with more disks on the machine , then scale-out with sharding.