What makes a 10x programmer?

There has always been a huge range of productivity and quality amongst programmers, and in the old days was estimated at a 20:1 range. In the field of the Arts there is more than 1000:1 range in skill; in fact, it is impractical to compare skill in the arts as it is subjective, but at an objective level there are obvious signs of superhuman capabilities, especially in music composition (which is a different kind of “programming”). Take for example Donizetti’s composition of “Don Pasquale” in around two weeks which is like writing 50,000 lines of code in 2 weeks, an incredible feat one of the greatest artistic achievements of all time. So 10x for programming doesn’t sound unreasonable at all.

An environment where peace and quiet are available is very important for productivity, but putting a lousy programmer in a quiet environment still won't fix their code. Having studied a lot of code, i have determined that great programmers all use the same fundamental technique. The purposed of this technique is to generate fewer errors during authoring and a more regular code structure that also makes it faster to write as the code follows a more regular pattern. Keep in mind that approximately 85% of your time as a programmer is spent fixing your own mistakes. If you learn to write in a way that has near zero errors, you will be 8x faster that you were before, so that is well on your way to 10x.

This technique is not taught in school or even in books. The technique can be roughly described as "complexity reduction". As N. Wirth pointed out in his famous book "Algorithms + Data structures = Programming", all programs break down into these two parts: Algorithms and Data Structures. The fundamental algorithms of most commercial programming tasks are pretty simple. 99% of us programmers are not building vision systems or trying to translate from english to other languages... most of us have pretty simple algorithms to follow, like “sort this table of sales results in descending order and take the top 10”. Also, most of the data structures we use are often fairly simple, like a table of transactions for the month. So given that the majority of business programming tasks have simple data structures and simple algorithms all the programs should be more or less identical when you remove the skin. Programmers build a bridge from the customer problem space to the final API of the operating system or environment that is being used. Some programmers construct a bridge that goes nearly straight between these two endpoints, and others create a more wandering path.

The best programmers minimize the number of variables, the number of layers, and through a gradual reduction process create less code, that is clearer and cleaner in purpose. This reduction process is fairly mechanical, and all the best programmers instinctively do it, while bad programmers don't see the reduction process. In the old days of circuit design, you would use Karnaugh maps to reduce the number of gates to express a truth table, the best programmers are doing the same exact kind of thing in their head to minimize the total amount of logic and layers.

Could we improve the skill of programmers? Can this be taught? Absolutely. But in order to learn you have to be willing to learn first, and a lot of people think that because they got a program working that it is "good enough". Most people feel that however ugly, long-winded, and tortuous the path was, is unimportant.

Imagine in the arts and crafts if execution of the raw task was only considered. So Barbra Streisand and Celine Dion would be underbid by sandpaper-voiced tonal disasters, because "hey, they sing the same words and get through the song ok". Thank goodness in music and art we have a lot of freedom to choose what we listen to, and talented people tend to get a large following.

Because so few people can quickly read the work of other programmers (usually because the code is so long and complex), we live in an era where code quality is invisible, and the sublime skill that exists in some people is completely unrecognized. Without recognition, how can training get anywhere? In bridge building, connoisseurs all admire the work of Robert Maillart. Look him up, that guy was incredible in the early 1900’s; a real fusion between art and science. The engineering fields don't get the recognition they deserve for the elegant, artistic work that exists. How about the Hetch Hetchy water system of San Francisco, that sends water 160 miles away almost completely powered by gravity? That is insanely clever. A modern wonder of the world if you ask me.

It is unfortunate that so few companies have the luxury of having two or more teams working on the same problem so that the best team can be identified. Usually we see only one team, and so objectively speaking one is often in mystery as to how good the team is. Interestingly enough, back in the days of Thomas J. Watson at IBM, when a critical task came up, he would typically put 3 teams on the task, and whichever lab won the contest, got an increase of staff and money. This approach worked very well for IBM, which was basically a monopoly at the time, and kept the internal teams at a very high level of energy. I worked on an IBM project after Watson's retirement, and by then the management of IBM had so deteriorated in integrity and guts that every team had to be a winner.

Another thing we see today is a fake 10x programmer, who copies and pastes large chunks of code from other programmers and stuffs his project with tens of thousands (if not hundreds of thousands) of lines of code that do some mysterious thing. So it is extremely foolish to measure productivity by lines of code, because people can create a monster code base in no time flat by merging modules that they get from other sources. I admire short, clear, concise programs that get the job done and are almost eternal in their purity.

Is quantum computing real?

Quantum computing is real. I would describe it as a real boondoggle. The definition of a boondoggle is “work or activity that is wasteful or pointless but gives the appearance of having value”.

The people selling quantum computing mention cryptography, and other things that seem plausible. When the NSA wants to crack codes, they just fire up a million core computer, and those don’t require any breakthroughs, just lots of hardware and a big room. Quantum machines have no practical use, nor will they for decades to come.

It is extremely fascinating to play around with superconductive materials. So I don’t blame researchers for using any excuse to obtain funding. At very low temperatures, metal loses its normal resistance to electricity, and one can produce phenomenally large magnetic fields, which then creates all sorts of exotic conditions that at room temperature don’t exist.

You have to consider quantum computing to be a side-branch of particle physics, which is a fascinating and baffling area of study. But don’t expect to see a practical product in your lifetime. It is most unfortunate that the area of physics that holds the greatest promise of improving mankind’s life on earth, low energy nuclear reactions (also called “cold fusion”), is getting negligible research money, and instead we spend billions on things like hot fusion which has zero likelihood of producing usable results. Hot fusion is predicted to work "in 30 years" for the last 50 years. 30 years means that is enough time to retire and die before they can assign blame for lying.

There is a fellow who is working on low energy nuclear reactions who postulates a lower state of the hydrogen atom he calls the “hydrino”, and proposes that the dark matter the astrophysicists insist exists is big quantities of hydrinos. Anyone who has studied the standard model has to admit that it is a very messy, unsatisfying theory, and doesn’t explain radioactive decay at all. There are many mysteries to still unravel, and any day i expect a major breakthrough that will cause the current standard model to put into the wastebin.

Is global mutable state bad?

This is one of the great buzzword phrases in computers today: eliminating mutable state. Too bad these faddists fail to understand how a computer actually works. A computer fundamentally uses a global mutable state, called RAM and CPU registers. Those are constantly mutating, and are globally shared. Yes, you can get into trouble, but all programs eventually devolve into something that updates global mutable state. Otherwise no work is actually done. How one can structure this inevitable changing of global mutable state so as to have the fewest programming errors is an area of great theoretical and commercial interest. But to characterize the basic operation of what must eventually happen as “bad” is somewhat absurd.

One might more properly phrase the question, “how can i reduce the errors in my program to a minimum, while still accomplishing my goal in a reasonable time frame, and with a finished product that can be understood by other people without them scratching their heads on how it works”.

There are languages like LISP and FORTH, that are in the family called transformation languages, that win every contest imaginable for program brevity, but i defy you to understand a large FORTH or LISP program! Very tough.


How do below average developers survive?

How do below average workers survive? Simple, we are human beings and pure performance has never been the benchmark for employee retention. Some people are fun to have around, some are loyal, some are owed favors, some are relatives, some bring out great qualities in others even though they themselves don’t appear to do much (the catalyst type of person)… you get the picture. Half of all workers are below average, so who cares? And who is some all-mighty judge deciding who is better or worse than another person? I am a terrific programmer, but no programmer knows every application area or toolchain. If you put a giant Ruby on Rails program in front of me, i would be clueless, because although i can work in a dozen languages, Ruby isn’t one of them, and it would take me months to get up to speed on it.

The real question one should ask is: how do i stay up with the constant changes without burning out? And how can i maximize the value of what i do know?

In a well run company, the systems are so well designed that an ordinary person can get the job done in an 8 hour day without heavy stress. That is the beauty of a well-designed company system. In the entrepreneurial discussions you hear absolute nonsense about how you should only “hire great people”. If everyone was great, your company would be unprofitable because they would be spending too much money on all these great people. As my dear friend J. Moon points out: Ray Kroc didn’t invent the hamburger, but he created an amazing system that delivered a consistent product at a massive scale, with ordinary people.

Why isn't the Haskell language more popular?

Please note that this answer is not a criticism of Haskell, but an impartial observation of the qualities it possesses, which explains why it is not in general use. We know that some people love it, but isn’t under serious consideration by anyone to replace Java or JavaScript, the two most popular languages today ( excluding MS Excel). And you could replace Haskell with any number of powerful languages that have been around for more than 20 years yet are still very obscure.

A computer language can be characterized in many ways, but the most important aspects are:

(A) power,

(B) brevity,

(C) range,

(D) efficiency in speed/size, and

(E) ease of reading and maintaining

power can be thought of as, can a program be assembled from smaller pieces into a powerful complex final product?

brevity is about how many keystrokes you type to achieve a result. often a powerful language has brevity, however, extreme brevity such as APL and LISP possess entails trade-offs.

range is about how many different kinds of systems you can build with the language. Real time communications systems? Video games? Car combustion computer software? Payroll? etc.

efficiency is about how many resources it takes to run the final work product. A combustion computer needs to start up very fast, and often has serious constraints on RAM and CPU power.

ease of reading and maintaining is all about someone other than the original author being able to understand and modify some code without breaking the system.

In the commercial world ease of reading and maintaining is the dominant concern. If you use an obscure language, not many people can read it. So people tend to use the same languages they did 10 years ago, and language preferences move at glacial speed.

Haskell is not great in D (efficiency), but really suffers in (E) ease of reading and maintaining. Otherwise it is terrific, and the people who only care about properties A, B, C can sing praises of Haskell until they are hoarse. However, (E) is the fatal flaw in Haskell, and large Haskell programs are exceedingly difficult to understand. It will never be popular, and just like LISP and dozens of other powerful languages, they will remain niche forever. There is a reason why BASIC, C, Pascal, and many other very simple languages are so influential, because they are pretty easy to read. Any increase in power at the expense of ease of reading is ultimately judged by the general public as an unacceptable tradeoff. A program is written once, and used for decades, and along the way will pass through many hands.

Please note that i am not considering the benefits to the programmer who may be seeking job security; there are many instances in the history of programming languages where the bulk of programmers selected the most verbose language available so as to increase billable hours, and rejected superior alternatives. In defense of Haskell and other advanced languages, part of the resistance is a realization that a more powerful and brief language would reduce overall billable hours.

Debugging - the subtle lie

"Debugging is like being the detective in a crime movie where you're also the murderer"  - Filipe Fortes

From the earliest times in programming, people have used the term debugging. At one point there was a moth in a relay, but programmers had to invent a term for what they spend most of their time doing, so it was inevitable that a word would be invented to describe the process of fixing your own mistakes. It is estimated that 85% of the spent programming is consumed fixing the mistakes put into the code by the author. I know of no other job where 85% of the time is spent fixing mistakes of your own making. Programmers imagine they spend their time typing in programs, and so people often quest after briefer, more compact notations. The language called APL was the all-time champion of brevity, but its cryptic notation ensured that programs were almost write-only; only the author typically could understand them. The reality is that the computer requires such inhuman precision that it is a struggle for us to program, because the tiny mistakes that are made, often just a few words, cause the program to act in baffling ways.

Any programming language that purports to be a significant improvement over current technologies must make error correction a much easier, smoother process. In fact brevity may work against reducing error. Simplicity, regularity, and the ability to understand other people's code is paramount in the nex† generation language race.



Stock tip of the day - buy Ford (F)

You can profit from my research and losses.

Ford is my stock recommendation. it is near the one year low, and pays over 5% dividends. I have lost a lot of money on it, watching it sink as if it was a doomed company. This month’s sales are 7% down from last year. Well, last year was the highest on record. Overall 2017 is a good year for them. Today the Tesla Motors stock market cap passed Ford. Tesla is now worth $48 billion, and Ford is worth $45 billion. 

Compare sales:

Tesla:  $7 billion/year  vs. Ford: $150 billion/year  (21 times larger)


Tesla $0.7 billion loss (10% loss) vs. Ford: profitable

Cars shipped:

Tesla:  approx. 100,000 cars/year, one factory vs. Ford: 3,200,000/year, factories in dozens of countries (32 times larger)

Ford sells products all over the world to the masses. It has always been a cyclical company, as it cannot escape the overall financial state of its customers. Tesla is a luxury goods supplier, and is fairly immune to the general business cycle.

There is nothing seriously wrong with Ford, other than they are an old company with lots of pensions to pay, as automotive companies became paternalistic and rather socialist. Their product line has been steadily upgraded. They just came out with an aluminum F250 truck, which is the largest selling commercial and utility truck. That business alone would make them a large company. 

Tesla is a full of hot air company. They have never shipped a mass-market product. They lose money every year. They pay no dividends. Since they have no previous track record they can tease about a fabulous new product without disrupting the sales of their existing products. In the software field we call this advertising of a non-existent product “vaporware”. Since Elon comes from the software industry he has adopted the same tactics, that of promising a perfect new product that is always a little late, except that it turns out to be a lot behind schedule. 

Tesla has no service network to speak of and cannot possibly supply all the parts and repairs for the 500,000 cars they plan to build by the end of next year. When people have a luxury car they don’t put a lot of miles on them; that doesn’t tax your repair and parts capabilities, because the average wealthy person has many cars, and travels around the world so much they drive their cars little, not to mention having multiple homes so the miles driven per car is very low. Compare that to a commuter or contractor who beats the hell out of their single vehicle. Did you know the average age of a pickup truck in the USA is close to 20 years old? Ford will be a huge company for decades to come simply from selling replacement trucks.

Tesla is a software company, and that is Elon’s great strength. As cars become more software oriented, Elon’s company will do well. Ford and GM have really lousy programmers, have never paid attention to it before. So they will struggle in this transition. however, you can always buy software from a 3rd party, and it isn’t hard to run new software. Don’t like your autopilot vendor like MobileEyes which got fired from Tesla after the crash? Just buy a program and run it on the same computer. 

Once you enter the mass market as Elon is planning to do, their cars will get dented and scraped, and the need to supply parts and service will skyrocket, and since Elon hates dealers (refuses to have a dealer network), his antagonistic attitude towards third party vendors will bite him in the ass. You simply cannot deploy mass quantities of a product without a dealer/service network. This is how Citroen, and the French totally failed in the USA. They didn’t build a network of quality dealers. I have no love for auto dealers, and their ripoff attitudes, but it is a fact that the parts warehouse of Honda in Reno, can get any part for any car in 24 hours to anywhere in California. 

There are numerous stories on the internet of people waiting months for a Tesla part. if that was your only car you would be furious. 

People are irrationally pounding on Ford, and over-optimistic on Tesla. I doubt you can get 5% on a bond which is only going to go down as interest rates creep back up. I expect people to stay pessimistic on ford until the model 3 is late, as it inevitably will be. Any one missing part will stop his assembly line, and ramping up by a factor of 10 always has unforeseen challenges.

also, i am calling peak apple, the minute they fill up their spaceship building with employees (which should happen around September), they can shut the doors because no new idea will get out the door, because no new idea will be worthy or sufficiently important compared to the majestic building it came from. 



The era of interchangeable parts

The two big challenges for the next gen computer language are reducing complexity and creating a system where interchangeable parts are usable.  There are many wonderful new languages like Elixir, Julia, Parasail, Go, Rust, Dart, Swift, etc., but... none of those languages actually tackle the 2 big issues of complexity and interchangeable parts. In fact, none of these languages even really tries. They are instead adding features on top of their foundation language, and merely improving a few areas. For example, Parasail does a phenomenal job of making it much easier to utilize parallel processing. But the rest of the language is fairly conventional. None of these languages will likely displace the current leading languages (JavaScript, Python, etc.) because the improvement is insufficient. If something is only 10% better than the existing technology, the pressure to change is very slight.  The mass of developers inevitably coalesce upon a standard that the group as a whole picks. History shows that the programming community as a whole selects the language that generates the most billable hours.  We have seen this before with COBOL, and Java. JavaScript is now #1 in terms of code being written, and as you can see from the diagram from the previous blog post, it requires a bewildering array of tools to make it even tolerable. It is time for JavaScript to die, and so the burning question arises: which of the new languages has the potential to displace JavaScript (and most of the other top 10 languages).

The current candidate languages/systems are:

1) Red,  2) Eve, 3) Elm, 4) GunDB, 5) Beads, 6) a few other languages which are in stealth and haven't shown their hands. Let's look at the first three candidates:

Red (red-lang.org) is an improved version of Rebol. Rebol is a LISP family language that has a much cleaner syntax. In some ways it is reminiscent of Objective-C. The beauty of Red is that you only need a small set of tools, and because each module is its own domain specific language, you can hook up lots of components together. However, the Red language doesn't have a database inside the language, and that means you are inevitably going to have a serious mismatch between the very flexible language and an inflexible database. If you dramatically increase the power of a car's engine, and don't beef up the transmission, the car will burn out the transmission. The database is like the transmission, it has to absorb all the power the engine creates. And this is a central flaw in the in the thinking of Red. If they don't include a database, it can't go much further ahead of the current languages. 

Eve (with-eve.com) is a new database system and language. It has inspiration MySQL, and goes way beyond it with an interesting free-form database based on entity/attribute. It adds a feature which allows code to depend on values in records, and to automatically redraw affected DIV blocks. The developers didn't like the problems variables and loops, so they banished them from the language. But banishing one of the fundamental operations of the underlying CPU never works in the long run. If you don't have variables, then people will put their variables into your database and ruin the purity of your system. The other great limitation in Eve is that they are still presenting to the programmer the HTML drawing model, which is terrible. Since most graphical interactive programs comprise 85% of drawing code (my own experience), if you don't fix the lousy graphical model of HTML you can't really fix programming. Eve is still in constant flux, and i expect they will backpedal and fix these problems due to user demand.

Elm (elm-lang.org) is a very interesting language. It was inspired by Haskell, but is a much more pragmatic approach. Elm has a very simplified graphical model, and of these entries has the most evolved toolchain. It does however present some difficult abstractions to the beginning, and is not that easy to read for a beginning, as you must have a tremendous memory to remember how many arguments each function needs. By eliminating the parentheses that LISP used, you then rely on the programmer's memory. Evan the author has a prodigious memory, and i suspect that people will find it a little difficult to learn. It doesn't really have a database system.

Gun DB (github.com/gundb) was designed as a graph database, but is evolving a general purpose language. Out of the box it offers an offline-first database that can sync with other peers. This feature alone makes what is a very tricky thing program pretty easy. Graph databases are the ultimate in flexibility. Currently the best graph database implementation is Neo4J, however, Neo4J has gotten sidetracked into connecting their system to older databases, and is burning up their development staff on a hopeless quest for compatibility which can never be achieved with the big database vendors. The Neo4J Cypher language is very weak, and you can't build programs in it. GunDB has not implemented the relationships part of Neo4J (its most interesting aspect), but it is a lot of fun, and the pragmatism of the author would indicate it will evolve into a useful thing.

We will talk about Beads in another post. It combines a graph database, with a simplified drawing model, all based on a declarative + deductive language that has a robust mathematical foundation.

One of the things that is built into Excel that is not present in any other language than Beads is the property of mathematical closure in the underlying arithmetic. If you divide by zero, what happens? If you multiply by an undefined variable what happens? If you take the square root of a negative number, and then multiply that by 2 what do you get? many of these languages don't have a precise definition of what happens, and this leads to program errors. The protected arithmetic of Excel is one of the main reasons the amateur programmers of the world, which number in the hundreds of millions, don't venture into conventional languages, which don't protect the user.






A clear example of the mess programming is in

A very enterprising Kamran Ahmed and associates has created a wonderful set of charts showing how deep and wide the mess is in development today. The amount of knowledge one has to have in order to be productive using conventional tools is absurd. The true next generation language will roll up all of these components into one simple tool. 

(for the full diagram see https://github.com/kamranahmedse/developer-roadmap/blob/master/README.md

Get ready to be sick of machine learning

There is a new programming technique now making the rounds, called ML or machine learning.

This is a highly improved version of a very old technique, whereby you can take very large data samples, and look for patterns in them. Everyone and their brother is now using these techniques to build artificial intelligence. Pattern matching, like matching a face in a set of photographs is indeed an amazing technique. However, matching people’s faces is not a prediction, or understanding. You will see this technique applied to stock picking, business picking, and innumerable business areas. The flaw in all of it is that it is like driving via the rear-view mirror. If you base your prediction on which stock will go up based on Twitter mentions, then people will create Twitter robots (they already have them) to automatically comment on a stock so it zooms up the mention leaderboard. Everyone of these parameters is subject to manipulation, and so it all becomes circular reasoning. 

In 5 years we will be sick of ML and it will have some spectacular mis-predictions. I have no love of statisticians running the world, but every day government policy becomes more statistical in its basis, and i find it deplorable. Creativity and invention are not statistical. They are an abrupt, unpredictable thing, and they have the power to transform the world, not the following of existing patterns, which a machine learning system can only do.

No amount of statistical work could create Beethoven's 9th Symphony. And if you studied his output you would think oh he will make 10 more symphonies.. but he was deaf at the time he wrote the 9th symphony, which makes it the greatest achievement in the history of the arts (the equivalent of Picasso making his greatest painting if had been blind at the end of his life), but a completely un-reproducible, unpredictable event.

When creative people make something new, the world benefits, and we need to stop worshipping statistics, which can only count what exists.

When computers are used badly

I have been programming daily for 46 years, which goes back to the punch card era. 

I have always avoided working in the military area, which has most of the interesting projects, but rarely improves human life. Computers have been used to grease the wheels of commerce, and their inherent honesty has greatly improved human life. However, when a dishonest person uses a computer for evil purposes, like Bernie Madoff, then it amplifies evil. The prevalence of computer viruses is another great negative.  

Unfortunately the negative uses of computers, to spy on individuals, and to permit governments to achieve ridiculous heights of intrusion and money extraction, is cancelling out many of the early benefits, and we are now in an era where local governments are using computers to punish the lowest strata of our society, and if you have studied markov modeling, i can show you my model of homelessness generation, whereby local governments have created a system that generates homelessness to such a degree that we are now surrounded by a beggar class, previously only seen in Indian cities. The escalation of a parking ticket from $35 to $450 in 30 days, and the $200/day after-tow storage charges are devastating the bottom segment, all driven by computers.

The increase of regulations has made it far more favorable to be a gigantic bank that can service the hyper-complex regulations, and the net result is that banking has been consolidated into loathsome, unresponsive, inefficient mega corporations, that are so far from A.P. Giannini’s philosophy that he is turning in his grave.

At this point, only an effort at simplification can help our society. Norbert Wiener wrote a pretty good book a long time ago, called “the Human use of Human Beings”, where he covered some of these topics. He was a very clever man, and a pioneer in what they called back then “Cybernetics”, which we now call “Artificial Intelligence” or “Machine learning”.

For example, all the wage taxes and regulations, including the staggeringly stupid higher minimum wages, all have unintended consequences. In Marin county, all of the jobs that teenagers used to be hired to do (work at the corner grocery store), were taken by 30 year old highly responsible illegal aliens. The unintended consequence was that Marin county teenagers never learned how to work. A very high minimum wage will have the effect of eliminating all young labor. Young workers are goof-offs, sloppy, and unreliable many times. But they need training, and many people learn a lot from their early jobs. If we have only high wage jobs, people will avoid hiring someone. The drive to higher minimum wages will result in higher teenage unemployment, and continue to exacerbate urban poverty. We as a society need to allow people who aren’t worth much to have a job and earn some money. In Japan they have their 90 year olds watering the sidewalk. No, the sidewalk doesn’t need watering, but they do it because it is a job, and it gives them dignity.

The great weakness of a computer is that it has no compassion, and cannot tell when it is time to relax the rules. 



The world in a clock

When one is comparing languages, it is fruitful to take a small example program, and implement it in multiple languages. The previous examples were a little too hard, and few stepped forward with example code of their own. So let's take it down to rock bottom simplicity. The written specification:

1) draw a standard 12 hour clock with 3 hands: hour, minute, and second. Show the current time in GMT. Move the hands by jumping from second to second. Make the clock look like this:

If you were try to program this in C / Win32 environment, you would find that it takes hundreds of lines of code, and reading hundreds of pages of manuals. You can't just draw on the screen, you gotta first create a window, and then create a graphic context, select brushes, etc.. Since Win32 had no concept of frame rates or animation, you had to manually create an offscreen context, draw into the offscreen world, then refresh manually. It was extremely hard to get a smooth frame rate. That very cumbersome system made Bill Gates the richest man in the world, because it was adopted by 95% of personal computers. That environment was a reliable but clunky platform to program on, but it took hundreds of hours to get up to speed. Professionals didn't mind, but amateurs sure struggled.

Instead of picking a language for this clock project, let's imagine the optimal language for this task. How short can the program get? What is the absolute minimum number of things you have to specify in order to get the result? Every program is information given to the computer sufficient to make it produce the result. We know that you must specify the following:

1) the center, radius, color, thickness of the circle
2) the color of the interior
3) the relative length of the second, minute and hour hands
4) the color and stroke width of the hands
5) the current time in GMT, converted to hours, minutes, seconds
6) the update interval for the refresh of the screen (1 frame per second)

so there are 5 colors, and 14 numbers that one must supply in the code. Let's assume the system will tell you the current time, so the OS will supply the 3 time numbers. So there are 5 colors and 11 numbers in your code (at the minimum). An optimal language will just have 11 numbers in the program code. However, there is no available language that can accomplish this task with just 11 numbers in the code, for the simple reason that library functions like sine and cosine invariably use radians as the units, and converting hours into radians involves additional calculations and numbers. Every time you add another number into your code, there is another chance for error. The numbers given above are really the specification of the program, and can be presumed to be correct. Any program that has only these constants, and no other calculations will probably have zero errors. If the only thing in your code is the specification you were given, then no errors can happen. This is the whole thrust of the declarative language movement: if your code is merely a reiteration of the specification, then almost no errors can occur. 

So the absolute smallest would look like:

draw a circle with center CX, CY with radius R fill with interior_color, stroke with border_color
draw the hour hand from at the proper angle, color and stroke thickness
draw the minute hand at the proper angle, color and stroke thickness
draw the second hand
refresh the clock display every second

So we can with mathematical precision, state that the minimal code is 5 lines, and should have 11 numbers and 5 colors. Since no general purpose language will be perfect for drawing clocks, we can get approach perfection. But how close can we get? That is a very interesting question. Our measurements show that Beads is the closest to optimal language that we have seen.

But step back for a second, and let's catch up to today's graphical standards. Nobody wants to see stripped-down programmer generated graphics. Let's take the same clock, and jazz it up with some bitmap graphics:

This clock has one more dial in it, the date at the bottom, So there are 4 hands, the day, hour, minute, and second hands. If you are given the background artwork consisting of the watch background, and the 4 hands, you should be able to draw a much nicer looking clock. This is far more representative of what people want to build. The previous clock is pathetic in comparison.

So now, the program needs at a minimum the center on the background image for the hour, minute, second hands, the center for the day hand, and then 4 more images and the origin point of the hand inside those images. The total information needed is therefore:

5 image file names (CANVAS, day, hour, min, second hands)
center_x, center_y for hour/min/sec hands in background image (2 numbers)
center_x, center_y for day hands in background image (2 numbers)
the origin xy point inside the 4 hand images (8 numbers)
a refresh rate of 1 tick per second (1 number)

This much nicer looking clock has the same amount of information in it as the stripped down analog clock! Even though this project is a 100 times nicer looking, it is basically the same program and should be no longer than the previous version. How short a program could exist for this project? Clearly it is 6 lines of code:

1) draw the background image
2) draw the day hand bitmap at the center with the proper angle
3) draw the hour hand
4) draw the minute hand
5) draw the second hand
6) refresh the screen every second

I look forward to seeing your submissions, which I will add to this blog and comment on. Given that the fancy bitmap version is only 1 line more of code (to draw the day hand), this should be a trivial project. You can download the graphics so you can try this task in your favorite language:


But it isn't a trivial project in many languages. Just packaging bitmap artwork into a program, and making available during execution is a trick. HTML5 can load images across the web, with no effort, but when you are talking about a product that can run offline, it gets much trickier. I assert that this small example shows so much of what is wrong with modern development systems, where the minute you ask for external resources to be packaged a whole can of worms is opened.






Avoiding inefficiency in multi-platform product web serving

In the previous blog post, we described a house picker test project. The program shows a list of houses you might want to buy around the world. When you are talking about a dynamic list of houses, one can assume that the list is kept on a server in the cloud, and that the client is downloading the house descriptions and photos from the server. But each client has a different resolution screen size, and that means the photos if they are served at one single size, will need to be resized at the client side. This will result in blurred images, as the browsers don't do a good job of image resizing. So the burning question is, what resolution graphics do you store on the server? If you put nice big images on the server, and the client happens to be a tiny mobile phone, you are downloading much larger images than you need. Some people will choose to store smaller images on the server, which looks bad on big screen clients, to make the mobile clients run faster. 

As an experiment, i made a CGI-BIN program that lets the client program ask for exactly the size image they need, and resizes the image on the server side, storing it in a cache, so that repeated requests for that same size don't need to perform the image processing again. It is a small program, and results in the perfect size image downloaded, with zero waste in bandwidth. It turns out web browsers cache CGI-BIN requests the same as plain old images, so the browser still works great. I calculated that it cut down the number of bytes downloaded by over 50% on a mobile client, because you don't need the full resolution of the graphic. It makes you wonder how much resizing is going on at the client side, and how much of the total traffic of the internet is wasted because graphics are the wrong size. Hard drive space is incredibly cheap nowadays, and bandwidth and server time is much more expensive. Downloading a graphic that is twice the size it needs to be, not only chews up bandwidth, but your server capacity is directly proportional to the number of clients you are capable of serving at one moment, and when your server is pushing down to a client something twice as big as it needs to be, you are tying up the server for twice the time period.

Not every website serves up images, but those applications that are image-centric would greatly benefit from this optimization of download. 

It is also true that there is a lot of plain ASCII JSON encoding being used in clients, and Joe Armstrong got very heated about that in a talk a while back, where he noted that Ericsson sweated hard to save every bit in their overhead, and how infuriating it was to see bloated JSON encodings in packets. Unfortunately most programming languages are very clumsy at binary packing, and encoding things into a tight binary packing can be lot of extra work in many languages. Luckily in Beads, there is the concept of a binary bit array, and you can encode variable length numbers on bit boundaries with trivial effort. Tight packing of packets can make your online service be a lot zippier, and not require such big server resources.

Application Challenge #2 - The pied–à–terre picker

Here is the once sentence spec:  "Build an app that lets you browse through international houses for sale, showing a photo, and the most important stats about the properties".

Here is a rough sketch. A good programmer can build the entire system from this:

the currency button at the bottom pops up the currency selector. the sort button lets you pick from 4 different sort orders.  The asending/descending button is a toggle that reverses the order.

the currency button at the bottom pops up the currency selector. the sort button lets you pick from 4 different sort orders.  The asending/descending button is a toggle that reverses the order.

Prior to programming, it is often very helpful to draw in Photoshop a mockup of the main screen, from which you can get approximate measurements of various sections of the screen. Here is the photoshop mockup:

Draw a photo of the house, the country flag in the upper left corner. The city, # of bedrooms and bathrooms is listed, along with the area, the price per area, the asking price in the national currency, and the converted price based on the preferred currency. To save space, prices are rounded to K or M.  If the currency selected is USD then show the areas of square feet, otherwise show square meters.

To assist you in this challenge, i have posted the resources of the house bitmaps, flag bitmaps, house data, as well as my implementation. To obtain the resources, download from:

Here is a link to the resources: 


This project uses a plug-in module, A currency rate query module, which queries the European Central Bank for the top 24 currency rates, which are published daily at 2:15PM in Frankfurt. This is the kind of small, but tricky little plug-in module that if offered at a reasonable price, people would be glad to pay for it. To use this plug in, you add one line of code: "PlugIn_Currency.get_rates", and 600 milliseconds later, you have the current rates. 

The other aspect of this project that is notable, is that there many unit conversions going on. You have a house, that is listed in British Pounds, but displayed in Euros or Dollars. When you switch to India Rupees, the numbers get huge, so you have lots of abbreviations needed. Do you really need to see 8 digits of precision when something is 4 billion rupees?  The USA market uses square feet, but in Europe they use square meters. In a computer language with physical units built in, there are no explicit conversions in the code; fewer places to make mistakes. 

Why aren't physical units in programming languages yet?

Can you name a programming language or tool that lets you write expressions like:

mylength = 3 feet + 2 meter + 1 inch

The web mentions ATLAS, but that language is long dead. I can't think of any recent language that offers this obvious feature in the simple syntax above. The Wolfram Language has made provisions for it, but it is ugly with lots of additional punctuation:

In[8]:=h = First[y[t] /. DSolve[{ y''[t] == -\[Gamma] y'[t] - g, 
       y[0] == Quantity[0, "Meters"], 
       y'[0] == Quantity[10, "Meters"/"Seconds"]}, y, 
      t]] /. {\[Gamma] -> Quantity[1/10, 1/"Seconds"], 
    g -> Quantity[9.8, "Meters"/"Seconds"^2]};

Adding units to programming languages was proposed in 1978 by Van Snyder of JPL. The Ada language committee considered it, but didn't get around to it. FORTRAN has had it on their "to-do" list for decades. What is it about this very simple feature, that confounds all the language developers? On the surface it looks very simple. We are asking for the computer to know some basic information about physical units, and automatically convert things for us. It sure would come in handy when you are doing gnarly calculations, and expect units of energy at the end, but due to some subtle mistake you have the wrong units which means your results are completely wrong. Money is a unit, and currency conversions are also in the "would be very useful to have" category.

In 1999 JPL crashed their Mars Climate Observer spacecraft due to a units mistake. That was a 327 million dollar project ruined by a simple meters--feet unit mistake. If only they had listened to Mr. Snyder and implemented units into their programming languages. Every day of the week, thousands of Excel users are making small and large units mistakes. Occasionally a massive error is exposed by the press. For an interesting read, see the WSJ blog about a $100 million mistake:


In 2007, the Fortran committee issued a report N1696, where Page v in the introduction stated the following:

        "The most common errors in scientific and engineering software,
        that a language and its processor might be expected to
        ameliorate, are mismatched, missing or excess actual arguments
        in procedure references, followed by out-of-bounds array
        references, and then incorrect use of physical units of measure.
        Explicit interfaces largely solve the first problem and help to
        avoid the second problem, but do nothing directly for the

Why has it been so tough to add units features to a programming language? Joe Armstrong, in his 2014 Strange Loop Conference talk, put it very succinctly: you only get one chance to do a language right. Once you have lots of users cranking out code, you will have a riot by your users if you break their hundreds of thousands of lines of code by making a major syntax change.

Make no mistake about it, adding units to a programming language is a major decision that has implications in the most crucial area. There is a lot of talk about functional programming nowadays, but at the end of the day, all of our computers, from mainframes to cellphones, are Von Neumann type computers, and do their work by loading values into a CPU, calculating a number, and then storing that number. This crucial action is performed in all languages by the arithmetic expression statement, and when you add units to expressions, you are changing the most fundamental part of any computer language. Just now Cambridge University has tweaked their FORTRAN pre-processor to add physical type annotations. A brilliant job of retrofitting a valuable feature, but also testimony to the glacial speed at which committees work. 

Unit-based arithmetic is one of the pillars of my Beads language. It promises to eliminate an entire category of common programming mistakes. In the end, programming is a creative human activity, like writing a novel, except that instead of tickling the minds of human readers, the programmer gives work to a machine whose sole purpose is to make human lives more convenient and enjoyable. The only thing stopping computers from massively improving the quality of life, is that it is so difficult to build reliable software. I have heard 100 startup pitches at meetings in Silicon Valley, where the sole purpose of their first $400k was to fund some piece of programming. They can't even begin to run their company until their software and websites are built; and they take too long and cost too much.

Take the "To Do" list challenge with your language of choice

I throw my gauntlet down, and challenge you to build the simplest graphical interactive project I can think of. Let's start with the concept:

Step 1 / Raw concept in ONE paragraph:

Let the user enter a list of tasks which are remembered. The tasks have a checkbox to indicate completion.  The user can add/delete/edit tasks.  The user can view the three possible sets: all tasks, not-yet-completed tasks, and completed tasks.  The tasks are sorted in order of their time of addition, but the user can choose to display them in reverse order.

Step 2 / Raw Sketch done on paper:

here we see the list of tasks, the mode selector at the bottom, the add and edit buttons at top. And the sort order triangle also in the header area. Entering text is done on top of the header. There is the concept of a highlighted row, and when selected, the delete row icon shows up.

here we see the list of tasks, the mode selector at the bottom, the add and edit buttons at top. And the sort order triangle also in the header area. Entering text is done on top of the header. There is the concept of a highlighted row, and when selected, the delete row icon shows up.

Step 3 / Design primary persistent data structures:

the data structures couldn't be simpler. One binary flag for sort order (forward/reverse), and an array of records holding 2 fields; the task name, and a flag if the task is finished.

the data structures couldn't be simpler. One binary flag for sort order (forward/reverse), and an array of records holding 2 fields; the task name, and a flag if the task is finished.

Step 4 / Draw mockups in a painting program like Photoshop

In this screen we show the basic layout. Mode selectors at bottom. fonts are identified

In this screen we show the basic layout. Mode selectors at bottom. fonts are identified

completed tasks are denoted with gray text and a red strikeout line. the active row is highlited and shows the delete button

completed tasks are denoted with gray text and a red strikeout line. the active row is highlited and shows the delete button

as a convenience to the user, the CLEAR button zaps all the completed tasks. It replaces the add icon when in completed mode.

as a convenience to the user, the CLEAR button zaps all the completed tasks. It replaces the add icon when in completed mode.

Step 5 / Determine keyboard shortcuts and user interface logic

A click on the Add task icon switches to text entry mode, where header is the text input area
keyboard shortcut:  INSERT key  (command INSERT on Mac)
ENTER key terminates entry
entry is limited to 100 characters

sort order triangle icon pointing up (default value) is oldest task at top (FIFO).
clicking triangle toggles to opposite mode, with triangle pointing down.

add button is not visible when showing completed tasks, is replaced by clear button

EDIT button is only visible if a row is selected

when text entry field has no text in it yet, show a gray entry hint "Enter task"

auto save changes every 5 minutes, and save automatically when quitting task

scroll bar thumb disappears when no scrolling possible (all items visible)

text will shrink automatically, down to 6 points. After reaching the shrink limit,
then truncate text until it fits, and append three periods ... to indicate truncation

only one item can be selected at one time. when hilited, the delete icon (red circle with X) appears

Art Assets
Noteworthy font
Roboto font
Application icons for each platform

Step 6 / Calculate key measurements for different device sizes

By examining mockups, and making estimates, you specify the key measurements that will be used. For Cellphones (with diagonal screen sizes 3 - 7 inches), you will have smaller dimensions. In this particular design, we are going for 3 different sizes, which covers cellphones to desktop computers.  The color specifications are extracted from the mockups, and are shown as hex color values, which are typically used inside the code.

By examining mockups, and making estimates, you specify the key measurements that will be used. For Cellphones (with diagonal screen sizes 3 - 7 inches), you will have smaller dimensions. In this particular design, we are going for 3 different sizes, which covers cellphones to desktop computers.  The color specifications are extracted from the mockups, and are shown as hex color values, which are typically used inside the code.

Step 7 / Coding the project

Write the program in the language of your choice. A good order of implementation would be to 1) create a dummy document, 2) get the viewing to work, 3) implement actions like text entry, text edit, task delete, and sort order toggling.  Start by building for the smallest display range first (cellphone, typically iPhone 4 which is 4 inches diagonal, about the smallest device that is popular). The larger the screen, the easier it is to fit things.

Step 8 / Testing

How many of the 5 main platforms can you deliver on?

1) Mac OSX,
2) Windows,
3) iOS,
4) Android,
5) Web browser. 


How long will JavaScript continue to dominate?

In five years JavaScript, and many other similar languages, will be considered obsolete. I don't mean gone, JavaScript will be around for decades because there are hundreds of millions of lines of code working fine, and not worth the bother of rewriting. What I mean by "considered obsolete" is that people will no longer prefer that language, and feel more productive using a different one. JavaScript is 20 years old and has numerous flaws that cannot be fixed without major surgery, and from a compatibility point of view it would be very difficult to correct without breaking hundreds of millions of existing lines of code. Node.JS is merely javascript running on the desktop instead of inside a browser; that isn't a technological shift, but merely a platform spread (if you consider as i do that the web browser is its own platform independent from Mac OSX, Windows, Android and iOS which are the main client platforms).  A language with a fresh approach will arrive, that uses a deductive model of computation. This next generation language will also offer very strong compile-time integrity/logic checks to assist the programmer in delivering reliable software. Reliability is the great unsolved problem in computer science, not the ability to handle large amounts of data. And by reliability, i mean errors in software caused by human mistakes. Computer hardware is incredibly accurate and repeatable, but look at our main operating systems: hundreds of thousands of unsolved, unrepeatable problems reported with no resolution in sight.

People are writing software at all levels today, from low-interactivity spreadsheets to 3D computer games. All of these tasks are plagued with bugs.  We are in a period of time like the early automobile, when your chauffeur was also your mechanic because the car was very likely to break down. Today software is being delivered, but you have to keep a maintenance programming staff around forever because you are afraid it will break down, and it takes so long to understand someone else's work that you dare not dismiss the already trained maintenance staff, even though they may be lightly used. This high-cost-of-maintenance paradigm has to go away, and just like in automotive history the mechanic has become much less frequently accessed. We have come a lot way in the computer business; it used to take 30 minutes of a skilled technician to just turn on the mainframe computer, and now our cellphone computers can update themselves and rebooting takes under 1 minute. But we also have a long way to go. 

JavaScript is the dominant language of the web today, because it was selected by a staffer at Netscape, and the other browser companies agreed to implement it with an acceptable level of uniformity. The entire HTML5 stack is riddled with fatal, intrinsic flaws, that no minor twiddling by the committee can possibly fix. To imagine future generations (5 years is like a distant point in the horizon in technology space), would continue to use the crude tools of today, is a grim prediction. I have a more optimistic view, that the current languages will be swept away by a far superior language and toolchain.

JavaScript considered dangerous

The JavaScript language is conquering the world right now. Google built a terrific compiler, called the V8 engine, and it is so fast, so good, that all sorts of applications are now being written in JavaScript. But the HTML 5 stack, which consists of 3 different languages, HTML, CSS, and JavaScript is an odd beast, and in the history of computer languages, not a particularly well designed language. It was selected by a staffer at Netscape 20 years ago, and highly similar to ActionScript, Adobe's language that underlies their Flash animation tool.  JavaScript has some really dangerous things it, that really bother a lot of people.  Example 1:

part1 = 10
part2 = 20
part3 = 5
total = part1 + part2 + part3
//total is now 35 as you would expect

part1 = 10
part2 = 20
part3 = "5"
total = part1 + part2 + part3
//total is now "10205"

In the second example, if any component of an expression is a string, then the whole expression is converted to string manipulation, not addition, which could result in baffling program malfunctions. This is an artificial example, but if the variable part3 is coming from some other part of a large program, and it just happens to be set to a string, the rules of the very dynamic JavaScript language mean that total could become a string at any moment. The inability to protect against stupid mistakes makes JavaScript one of most dangerous languages ever invented. When you write JavaScript you have no assurance that your program is correct; you have to test it extensively to verify it is okay, and even after testing, bugs can still remain.
The task of programming is difficult because it requires an inhuman amount of precision. When a language lets you get into trouble easily, it makes the whole process harder and more troublesome. The committee in charge of changing JavaScript knows this, but they can't easily fix it without breaking all the programs that exist. The author of Erlang, the brilliant Joe Armstrong, has noted you only one chance when designing a language to do it right, at the beginning. 


Apple caught with their pants down

The other day Apple released an absolutely terrific product, the iPad Pro. This is the king of tablets - with a price to match - and it further blurs the boundary between a tablet and a laptop. Along with the Microsoft Surface Pro 4, these two devices are solidifying a new category of device. But Apple has a very rigid user interface for their devices, and they shipped the iPad Pro with a huge flaw, which is the icons are spaced too far apart on the home screen, and the iOS interface doesn't take any advantage of the huge screen real estate.  The space between the icons on the iPad pro's home page has more pixels than the original iPhone screen!  Even though this product was started over a year ago, Apple could not get even a simple change effected on their operating system. 

courtesy http://www.macworld.com/article/3004823/ipad/5-things-i-noticed-in-my-first-hours-with-the-ipad-pro.html

courtesy http://www.macworld.com/article/3004823/ipad/5-things-i-noticed-in-my-first-hours-with-the-ipad-pro.html

Apple has tried to keep the number of screen sizes to a minimum, but market forces are pushing them towards a wider range of screen sizes, but the way you make software in the Apple world is to use their design tools, and those tools weren't designed to handle that wide a range. Adapting to a wide range of physical screen sizes is a major unsolved problem in software today, and you can see how tough it is when the mighty Apple with limitless funds at its disposal, couldn't fix the operating system home screen to make it show more icons. The size difference between an iPad Mini and the iPad Pro is so huge, that the one-format-fits-all approach Apple has been using has broken down.

In the Android world, you never know what size and shape device you are going to encounter. There are screens that are long and narrow strips, in the shape of a Hershey's bar, there are circular screens with a slice cut off the bottom (Moto 360), and dual screen oddities. There are over 12,000 different Android devices, and not one of them has more than 10% market share! Making software that fits itself like a liquid into any shaped container is a major challenge, and certainly one of the most important goals of my Beads project is to make it easier to write software that makes freely flowing interfaces.


The above chart shows the market share of each android device. This chart is from a few years back, it is much worse now!  graphic courtesy of OpenSignal.

Sooner or later, Apple will need to acknowledge that their software system needs to accommodate a wider variety of sizes. Eventually they will end up with a product in every size range. Look at Samsung's product line and their sizes. Basically they have every size imaginable, and the challenge to the software designer is to make things look good at every size. When Steve Jobs was around he insisted on only a few device resolutions and aspect ratios; Jobs was like Henry Ford who used to joke that you could have a car in any color you wanted as long as it was Black. But Samsung is like General Motors which offered cars in any color. Note that this diagram is obsolete, because Samsung has an 18 inch tablet now! And Nabi offers at 24 inch tablet for $400. It is a challenge to get software to work well on all these crazy devices.

Samsung device size chart, Image from http://tech.sina.com.cn/ 

Samsung device size chart, Image from http://tech.sina.com.cn/