Welcome to LjGww's Blog

Here we will see...

Pages

Sunday, March 22, 2015

Compile!

It turns out that I write blog once a year (or so). That is a fact found by my access to this online place.
Maybe it is just the time of the year to do it. Last posting was in february last year.
Is it a yearly checkup? Looks like new year's resolution it's just that it is not new year :)

So what has happened in the meantime?

1. I jumped jobs. Back to software development - kind of.
2. then, attention goes to kind of serious CMS systems (by now I've seen many serious CMSes so I am wondering what's serious?)
3. Had a somewhat change of heart regarding programming languages. Last time this has happened was some nearly 10+ years ago when I went to scripting.

Let's go thru!

My first impression recently is C#. I have been successfully avoiding Java and C# for quite a long time. As usual one can ignore mainstream only so much. It turns out in my life and perhaps this is some kind of rule for others - whenever I do Microsoft I get money (from IT).

On the other side when I do not do Microsoft then either I get no money or there is much more money. After three decades (and some more), it looks to me that Microsoft steady supply of problems create stable source of money. If I do not do Microsoft money is volatile. For the time being, I enjoy more or less steady inflow, however then, I do not have much time to spare for my interests.

Wisdom is always in the balance.

When I jumped jobs, original idea of that move was to join C# developers team. I did not know C# in detail, we have just been acquaintances. I could code in any C like language but did not know many things and C#, or better, I had to learn a lot of things in Visual Studio (I was avoiding that too). I had to learn quick, and my quick learning of C#  resulted in some GUI and some command line tools. Quickly it turned out that I could be much faster in making tools in Python than in C#. C# did not give me any particular advantage.

Development process is complicated when you are using objects that someone else created, notoriously tricky-to-find-answers Microsoft documentation is as usual only of partial help, deployment is scratching your head around - what is the best option?

The only thing that C# really gave me is its nativeness on Windows platform. Using C# to develop GUI app proves to be tricky and with a lot of caveats, some hacks in order to achieve something simple (like getting window of interest in front). There are 23000 modules or something like that, which in a way beats a purpose of re-usability idea. Jargon and over-engineering in the sense of modules design is rampant, it looks to me as there was a lot of selling and politics behind every module - simply put, whole thing could be much more simple and more efficient not that for every small thing (structure, method) you have PHD dissertation of object patterns. Its tiresome to develop new things with it especially when you need to research every single lib or object to use. It's good when someone already did a research and selected among thousands of objects some that are useful and made adapters to use. Bottom line - its a struggle (see two paragraphs below)

Putting above aside I must admit that C# is better Java than Java. C# is in fact very nicely assembled language with many nice features and on that note I challenge you to try Mono Develop (http://www.monodevelop.com/) and be amazed how good this thing is (not on Windows :) )  I have even managed to make MVC running. People frown on me (why you do windows thing not on windows), but take note that Microsoft realized that has to grow out of windows environment (http://www.wired.com/2014/11/microsoft-open-sources-net-says-will-run-linux-mac/).

That's all nice and dandy but, C# & Java are still C# and Java... something that you need runtime library properly installed that comes in several versions and then you have app that uses one, app that uses another and on general term at the end you have mess of a system that you should not touch because everything that has been fine tuned will crumble down because you do some upgrade. And Microsoft does upgrades often, everything breaks, then it needs fixing, and then I get my money :)

Sooo I am making tools and I realize, if I make tools and have to have run-time library then I'd rather choose something mature and well defined and more over - cross platform - so I choose Python or Ruby. Ruby recently got out of fashion, main reason some claim that Ruby is not any more good IO for the web and Ruby became famous because of Rails. It turns out that Rails (or what would be in Python world Django) are very fine web systems but with limited scale scope. Nowadays it is C10K problem (having thousands of requests to the web server so web library has to be able to scale and be efficient) which both Ruby and Python are natively not - they are single CPU. Yes you can do threads in Ruby and yes there are several multiprocess libraries in Python but multiprocessing is not for what these languages have been designed. Thus, they are good for tools. And I use them to make user tolls and tools for myself. I have inclination towards Ruby rather than Python, simply syntax is in my opinion better, but Ruby do not get nearly as much attention as Python does. There is much more support for Python than for Ruby but when it comes to moving things around between Linux, Windows and Mac OSX these two languages are very good choice.

When making tools I realized that coding and assembling tools is tricky not because of the design or language ability or even selection of available libraries (example: try to make excel sheet programmatically by processing some data it is contrary to belief very complicated without good library and many workarounds). It is not tricky because of having and not having library or solution, but it is tricky because of deployment. I faced that if I use scripts and have Python/Ruby installed on my PC that is not a case around, furthermore in enterprise environment you cannot just go around and install 'system' libraries and applications just like that (just like you can do on your own PC at home).

So very next things I search for are:
1. Can I make executable that does not need installation conundrum?
2. Do I have GUI posibility?

Reasoning behind point 1 is: Even with native C# people may not have (and often do not have) a .net installation I made tool on. Or... How I do make exe from Python and/or Ruby.

Second part is not too complicated, both Python and Ruby has something called freezing tools. You simply (take 'simply' with pinch of salt) compile current modules you need and it makes you together with some systemic DLL's an executable program. It is not elegant as java's .jar or .net's exes (figuring necessary things for .net is more murky). But even when you get there, beware of some caveats. For example py2exe that makes Python scripts into executables is not always working as it should. I use some web libraries that refused to run with with py2exe also recently I am trying to make GTK2 running across platforms and py2exe was failing big time to produce me an exe. Luckily there are some other freezing libraries around to play with, it's just time spent or required, that is not always available. Also note that you may hit the situation that your script may be 'exe dependant'. This means that it would need to be different code for interpreted and executable build runs. So you would need to detect if code is running as script or as executable. Better freezing libraries provide this detection so you can put it in the code.

On the note of GTK2, it is gnome GUI library built for Linux, but it started creeping around to windows and OSX. Alternative GUI library from older times is Tcl/Tk. Tk library is fairly supported across lin/win/osx but it is somewhat limited if you wish newer looking GUI app, Tk is working really fine but it looks old fashioned. GTK now has decent tools to design UI (windows, dialogs), that you import (comes as XML file) into your app code so you do not need to create whole UI manually in the code, just use definitions and write action code. I have GTK running in Python on lin/win/osx and this is for my needs fine intermediate GUI. If you want more than GTK offers, look at QT side.

This in general wraps up one area of run-time libraries and their tools. I will have no rest here. My curiosity stretches more out of so called 'virtual machines' - programming systems that create a pillow between your code and the rest of the system. Lets put it this way - if you are writing a business application Java, C#, Python and Ruby are just fine tools, get to the database (virtually any type), do some calculations, selections, move things around, template results (in good MVC fashion), create some html or pdf reports, make configuration, logging, command line options etc. We did not get out of what we normally do (input, processing, output). However, sometimes I do need to do systemic things, I do occasionally need to provide user a GUI, and I do need to do DEPLOYMENT, so, I started looking around where we are with compiling?

At this point I will jump over node.js because Javascript is something I do not want to get into here, albeit it deserves a look over of what is happening in that corner of scripting. Target here is compiling.

Also I will skip an overview of so called 'functional programming' with many players from Haskel, over Erlang to Rust.

C is still strong all around. In fact all of Python, Ruby and many others are still written in C (don't know what .net is written in, but I am almost sure that Java is also C based)... why C is not going away? I started using C in some 1980ties, this is now, what?, 3 and a half decades, at that time (once upon a time) with small memories, non existent HHD's and low cpu speed, C was smallest option short of assembly languages. Yet, C is still bottom line for everything everywhere. In all of its virtues I find C a bitch to code in. I just do not want to go back to C to be able to compile and make something systemic. So why we do not have alternatives to C?

But, we do have alternatives! Apple was pushing Obj-C and recently Swift. I just do not see this going elsewhere out of Apple, lest being cross platform. True enough gcc can compile obj-c elsewhere (mainly Linux)

Also we have pascal, fortran and C++, all carrying historical burden of developing computer systems focus philosophies (read: every fashion in what was or what is current computing over years has been somehow pushed to these languages) they become not fun to use. They are still good options, it's just why I would go back to 20 or 30 years (or more) old languages? I surely do not want to write begin end code blocks of pascal (modula), I also do not want to end every obviously completed statement with ; and to some degree enclosing code blocks in { } becomes tedious too in modern times. Also over engineering in object analyses and patternization infested OO concept. Functional languages frown on Object Oriented programming saying this is not scalable enough because data and code is bound together. Pouf! Overnight realization that modular programming and decoupling of code and data is a good thing makes 20 or so more years of OO somewhere overly focused on thing it did not deliver (re-usability). It's not that we haven't been warned. There were people who had rants on OO once upon a time, but noone really listened. OO was everything and everywhere, selling like hot cakes. Scaling up and movements in computer industry had its toll. OO is not any more the big thing in programming. Nowadays big thing in programming is functional programming, deep learning algorithms, machine intelligence, big data, massive parallelism, virtual and clouded OS boxes. So...

we come to DEPLOYMENT....

What was once upon a time an installation program now it is app store. Installation is kind of update to something you already have. In enterprise it is normally not possible to run some installation everywhere, OS updated are controlled, putting something in Windows and Program Files is no-no. So how do you deploy non systemically when almost all installation is systemic ???

There we come again to plain old C. Compile monolithic application or compile application and provide critical DLL's (or .dylib or .so) and you are good to go in what is called user space - where user can do whatever on his/her own.

So I would jump out of my skin if there were not two nice developments around. One is weird and another is weirder, but both are logical.

First option is Go (or if you search internet golang). Made by Google for the same thing I am having problem - deployment. Go is modern replacement for C. Advantages are many among others it has built in concurrent programming - result - twitter process billions of sessions daily (https://blog.twitter.com/2015/handling-five-billion-sessions-a-day-in-real-time) note that previously twitter used to be based in Rails (ruby). Now it is golang - why? Because it is small, easy to deploy, compiled, scalable and IT IS NOT OO at all :)

After four years of existence Go has delivered its promise. When I have first tried it, it was not yet version 1. Since then it come to 1.4 and have a multitude of libraries for most important things INCLUDING GUI based tools and it is CROSS PLATFORM. So you'd say Go is the way to go - it is. It is very good option. And you would learn also that there are some provisions to make language not bothering the programmer (many things like {} or () or ; are not any more required to be written everywhere as a must) - I think finally programming language designers do not suffer of programmers that do not know programming. Many programming structures have been introduced to teach people to program, but today you more maintain the code than learning how to program so we know how to make structures and program, why you are still treating us as students. Go jumps this hurdle - it is very pleasant language to use, yet still dense like C. Go ahead and try it out. (https://archive.org/details/TheWayToGo & https://golang.org/)

Yet, this is not all what I planed to tell you. There is something else that has caught my attention very strongly. If Go is somewhat weird and unexpected but logical outcome, Nimrod is weirder but funny enough perhaps even more logical outcome.

Think in following terms: Original idea of compiler is to compile language into assembly code or directly to native code. Virtual Machines compile to meta languages. If you have heard of Dart or Coffee Script (also Jade or less.js), all of them cross compile from one language to another. Dart and Coffee Script result of compilation is Javascript code (or in the future something called today ECMA script http://en.wikipedia.org/wiki/ECMAScript). So, if you can cross compile to scripting or CLR (http://en.wikipedia.org/wiki/Common_Language_Runtime) why you cannot use something more basic like C? Nimrod, which changed name to Nim does exactly that. It's code is cross compiled to C then compiled to native executable. This is ingenious outcome of pushing the paradigm of higher language treating meta-high language like C as low level assembly language.

Nim (http://nim-lang.org/) does not stop there. Apart from compiling to C it promises to cross-compile to C++, Obj-C and JavaScript and all of that to be cross-platform.

What Nim brings to the table?

By being able to cross compile to multitude of basic systemic languages it also allows using a ton of C, C++ libraries for everything under the sky and on top of it. This makes it extremely attractive as Go C interfaces are a bit trickier to use because Go has (extremely fast) native compiler. Nim's ability to compile to C also bring C speed of execution. A good introduction of Nim's abilities can be found on Dr. Dobbs Journal (article was written by Nim's author) - http://www.drdobbs.com/open-source/nimrod-a-new-systems-programming-languag/240165321

There is a small warning here. As Go can be compared to Python in the terms of support, then Nim can be compared to Ruby. Both Nim and Ruby are 'small' / personal projects. They are not supported by companies and/or foundations, business models etc... In my view this do not make them less valuable, on the contrary I get them as fascinating example that humanity often ignores what is best and settle with mediocre solutions. This is my contribution to it - use them in spite that someone tells you: you shouldn't use them because of: (pick a list of criticisms on why one should use some well known tool and not a small personal tool)

Use them and support them (Nim and Ruby), they are fine tools.

This concludes compiling and deployment section.

I also wanted to put couple of thoughts on web programming as today most significant things are on the internet.

Getting in the waters of C# & .net I could not miss .net CMS or frameworks. Until last year I was mainly ignorant of what happens in .net internet world. My interest was in open source frameworks and CMSes and there are plenty of them. When I am about that, apart from still favourite Slim framework (PHP) I also find Ruby's Sinatra (http://www.sinatrarb.com/) very attractive, much more attractive than far more renown Rails (most popular Ruby Web framework). Perhaps because for me web is not about web sites but about data one pushes thru the web system. Sinatra and Slim are fine tools if you are looking to make web service (or web application for that matter).

I was not much interested in Java app servers or anything .net, but I have encounters with two significant players in .net CMS world.

Before I dwell there I must touch .net MVC. As any other similar MVC it consists of database layer, controller layer and on the other side presentation layer with the template engine. Microsoft web is given in two concepts - one is 'old' ASP system which not anymore based in COM approach but sitting on top of .net platform, so that one can write in multitude of languages that is available for .net (including for example Iron Python, version that compiles to .net)

then there is a second system, more modern. called MVC (now in some high version somewhere around 6, yet, I believe many use previous versions e.g. 4).

There are two nice things that are now provided in Microsoft web - both totally unknown to me before. One of them is Linq which is nice movement of decoupling database system from the web system. In ASP database was suggested or inclined towards Microsoft Access and MS SQL server and there were objects specifically available for MS databases.

Microsoft db systems are still present in the web game, but Linq makes a middle layer and supports other databases such as Oracle. MySql, Postgres and more (if you find drivers). Today data is everywhere, therefore ignoring other players do not work any more even for die hard Microsoft developers. Linq looks like SQL but not exactly the same, it looks more like active record objects. But decoupling is good idea nevertheless.

Another subsystem on other (view side), curiously provided as open source too. is called Razor (https://razorengine.codeplex.com/). Razor is very capable template engine and very much like many other templating engines. I am not too much interested to go in more detail here as you may read a lot about that elsewhere just to mention because it is important in the light of CMS systems.

Now, since we established that there are decent web cooking ingredients in the form of components and frameworks question becomes is there some CMS systems over there? And there they are, in fact, there are several .net CMS systems available. Among them some known from open source world like Nuke - so for example there is dotNetNuke which is free and you can try it.. .net web cms ecosystem is not too large. There is not a lot of things to choose from and plenty of them are not accessible until you buy them which makes hard to learn about them unless you are commercial client.

My experience covers sitecore (https://www.sitecore.net/) and sitefinity (http://www.sitefinity.com/). Main difference between them is that sitecore is primarily CMS where you can add components. It is european product (originated in Denmark) from one small company originally interested in marketing so whole system is basically a marketing tool with an idea to push info to customers via marketing channels one of which would be a web site. It is more understandable to a marketing-business person than it is understandable to a web programmer. Web panel is made for ordinary user and geared towards layers of decision makers. If you are one or couple of man band this multilayer approach gets in the way. It is possible to avoid it but generally if you are company they would be likely present. Although it is possible to develop components for sitecore it turns not to be as easy task. This is engine to maintain your web site in all Microsoft fashion you can imagine, that goes to that length that control panel looks like you started windows desktop all together with start button. These people really did listen to the customer - customer knows only about windows, we make it look exactly like windows. In marketing world not reinventing the wheel counts. Sitecore is significant player in the web site management field.

On the other site Sitefinity is a web developers wet dream - CMS is a component in MVC system. Telerik, a company that produces Sitefinity also sells web components for other web development work. This means sitefinity is more geared towards software development where CMS is support. It does not make CMS basic at all. On the contrary CMS component is very sophisticated and with modern design concept.

Both CMSes are very complex, and it is not uncommon that you may need training to understand nuts and bolts and how they function. I am not out of criticism about them, but here, I will skip that. Maybe once I would write thing or two about them again. Or maybe I leave them to the history.

I came to web section to go for something new and fresh to tell you. One idea that has caught my attention as almost ingenious. Surely there is a stroke of an inspiration behind next system I want to mention.

In the web programming we do think in terms of dynamic pages. Dynamically built on a server side or dynamically performing in the browser or both. Server side is primarily important and many languages and frameworks are built for the server side as to either serve pages assembled on the go or to provide dynamic data to Ajax in case of a browser application. But, what if one do not need automation at all at the server side? You'd say - Hey who does static web sites today ???? I hope I got your attention.

Even dynamic server side in order to fulfil client demand is put in cache. This means pages are often pre-built until data changes. Main question here is - how often you need to build pages?

If I run this blog site and change it every day this may warrant blogging application, but here I write once a year. Why you need to run app every time when you want to see a page here? You do not and you are served from cache. So what if I do not want to run any application on a server side because I can generate content and publish it?

Idea is not new. There were in 90ties so called web site generators. At that time web automation was very basic and things have been done through CGI system (I use to write web programs in C and do SSI - server side include). But think how you should do it today...

You'd have many formats for data, much of automation you can do on client side, you have templating engines so changing of design is easy. Putting all together you get:

Hugo!

Hugo (http://gohugo.io/) is moden 'static' web site generator with a twist. You write command files and then run generator that builds for you a web site you can upload to ANY web hosting. ANY is capitalized, because result for hosting online is pure HTML. Hugo does not require database support or programming language. Since it is HTML code is served instantly because it does not have to be built on the go, so, it is as fast as your web server and you can scale it up as much as you can scale static web serving (limitless). All hosting supports static web hosting so your site can be moved from any provider to any provider. There is much less security concerns since you are not running automation on the server side. And so on, there is a lot of advantages of static web pages.

All of above does not cost you in flexibility - when you wish to change your web site you add files, data and templates and you can have a new article on the web site or new design.

Hugo is great inclination going out of mainstream web school of thought. I guess it has bright (and opinionated) future for many applications (blogs, documentation online and such - things that do not change a lot over time or has relatively infrequent updates).

One would say - how then a blog built on Hugo can have comments, voting, liking etc? Luckily there is a ton of web service component providers where you can borrow the functional part. These are providers like Disqus and social networks who will be more than pleased to turn your user to access their system and if possible some along the lines charge you too. I can see them pushing hugo too.

When you consider making next web site, think in the terms how often you need to change it and try hugo - its awesome.

This is it for today. It is long enough. I hope you enjoyed it.

Post Scriptum...

The thing I did not make reflection on but that has profound impact on me was deep learning algorithm see more here: http://www.ted.com/talks/jeremy_howard_the_wonderful_and_terrifying_implications_of_computers_that_can_learn?language=en












1 comment:

  1. Post time of writing this text I have discovered two interesting languages that compile via C. This is similar to Nim language in concept and came a while ago. These are:

    - Vala (https://wiki.gnome.org/Projects/Vala)
    - and its counterpart Genie (https://wiki.gnome.org/Projects/Genie)

    in fact the same 'compiler' compiles both languages. The difference is that Vala is more like c# and genie is more like Python, the result is the same. Native environment for these two is Linux since they are designed primarily for gnome environmenthowever since these languages are using glib (http://en.wikipedia.org/wiki/GLib) and glib itself has been ported to Windows, OSX and mobile OSes consequently these languages transcend only Linux environment.

    ReplyDelete