The article seems to assume that there is something "wrong" with running on 25M lines oF Fortran. It does not provide any technical reason why Fortran is not the right tool for the job. Why does this need to be "Web 2.0"?
I'm also confused about this. I don't understand what is wrong with Fortran. In school we learned Fortran 2003 for numeric simulations etc. For my master thesis I programmed in C and there were a lot of things in C that I missed from Fortran.
1) Fortran is easy to learn, no need to know pointers etc
2) very math friendly, easy to extract subarrays/submatrices etc, no need to pass around array lengths etc. matrix's are really easy to deal with. I remember in C we ended up rewriting all two dimensional arrays (matrices) to column major arrays cause it was a pain to deal with (or maybe we were noobs)
3) useful built in functions such as matrix multiplications, in C you end up writing a lot of functions to do what seem like very simple things
4) Great math libraries such as BLAS/LAPACK
5) Seems to me like it less error prone than C (spend a lot less time debugging)
This is just my experience with it. Somewhat to my surprise I really ended up liking Fortran, and the new code doesn't look at all like the old code. For what we did it was also slightly faster than C. If I were to write a numeric program I wouldn't think twice about what language to use. I feel like Fortran is an undervalued language for calculations. But then again, I am by no means an expert.
> Now, if Bloomberg's code is a spaghetti monster (I have no idea), that is a problem. But that is a problem of legacy code, not a 'Fortran' problem.
This I think is the key point: lots of time when I've seen people mistake a practices problem like this (that result in expensive-to-maintain "legacy" code) with implementation language problems, they end up doing a conversion to a new language (often doing the worst thing you can do with a poorly specified, poorly documented, but basically working codebase, a "big-bang" replacement rather than an incremental one) without fixing the process issues, and so end up creating a new poorly-specified, poorly-documented, often less-well-working "instant legacy" codebase that just happens to be in a currently-fashionable language.
I client of mine still has a fairly large piece Delphi of code in production and I must say it's a bit of a liability for them. An example of a technical problem they face is getting the particular version of Delphi they use work with TLS (and that's becoming a problem now as everyone's starting to disable SSLv2 & SSLv3). The fact is that they simply wouldn't face this kind of problems if they have ported their code to something like Python, C# or Java.
On the non-technical side of things, they also struggle with hiring developers who are capable of (or willing to) work with Delphi, especially junior developers. A significant portion of their staff also doesn't know how to work with Delphi, so reassigning resources to match the needs of the business is often a challenge.
I imagine these issues would probably apply to Fortran as well.
Fortran is entirely modern. Last standard is what, 2008? And any compiler will just eat any previous version without complaint. You can take a huge swath of '77 code, put some 2003 and 2008 in there, and it will all just compile.
Python? Good luck. Every frigging month (practically) it's a breaking change. You have to work furiously just to keep your infrastructure working, or purposefully draw a line in the sand (2.7!).
I run Fortran every day (via NumPy and other packages), and some of us are investigating just switching to it for some projects (we need the math speed, and all the C++ matrix libraries involve compromises of one form or another). There's no reason I and my peers wont be doing the same in 50 years. It works, its fast, its debugged, and it is easy to use things like Python 5000 (by then over 70% of people will have made the transition from 2.7 to the 3.x version!!!) to do all the glue work that Fortran is not optimized to do.
Fortran codes ain't ever going to be a liability[1]. Python 2.7 on the other hand...
[1] because of being Fortran. Poor coding style was endemic in Fortran77, and that can be crippling, but to a large extent that is language agnostic. I acknowledge the required short variable names of 77 doesn't help, but people would not comment what the variables mean, and that is the real problem (IMO). Poor structure is another problem (masses of gotos). Yes, 77 doesn't make your life easy there, but discipline is possible. We can quibble about all this; my point is the 'modern' languages are going to be old quite soon, and we will face real porting problems then. Move 20,000,000 lines of JS and Python? Good luck.... Get a Python 2.4 code base working with 3.9? Ah, hmm, well .....
But Fortran? We build it and run it every day, easy peasy.
Python? Good luck. Every frigging month (practically) it's a breaking change. You have to work furiously just to keep your infrastructure working, or purposefully draw a line in the sand (2.7!).
What? There was exactly one breaking point, from Python 2 to Python 3. Which "monthly breaking changes" are you talking about?
Probably not the language itself, but the quivering mass of libraries with moving APIs and bolted-on design fads. Numerics is numerics--functions, matrices, linear algebra and indexing isn't going to change any time soon.
That Fortran gets bashed and Lisp is worshiped seems quite incongruous.
I'm not a Python developer but what I've seen and find frustrating is the slow migration of libraries to Python 3. Some projects have switched to 3 and some are sticking with 2 for now. Some applications are using 2 because they have dependencies using 2, but as those dependencies switch applications have to deal with combining Python 2 and Python 3 dependencies.
I know each issue is easy in isolation, but we're talking about old 25M line codebases here. If such things exist in Python it can be a maintenance problem.
I'm in favor of languages making breaking changes occasionally but there is something to be said for languages like Java or Fortran that are intended to keep compatibility forever.
"There was exactly one breaking point, from Python 2 to Python 3"
One breaking point?
The majority of the libraries that I use don't even work properly in Python 3. The standard Mysql library for instance, which is used in the majority of ORMs (including SQLalchemy) isn't included in Python 3, which means you need to make major changes/try to use a different library.
The case of many standard libraries were changed, breaking lots of code. Example: ConfigParser to configparser.
Even simple things like print 'test' is now print('test')
These are only a few examples. There are lots of other major changes that break 2.X code.
I have been using Python 2.X for all of my projects and wanted so badly to change to 3.x for my new projects. However, because of all incompatibilities, I can't.
I don't think I'm alone on this either. Python 3.X still isn't being used by the majority of developers.
(we need the math speed, and all the C++ matrix libraries involve compromises of one form or another).
Are you using lots of really tiny matrices? I would think pretty much all the libraries could just be linked with a good BLAS distro and then it would not matter what language that you use. Personally, I'm a big fan of the Armadillo C++ library.
Delphi could be considered a proprietary language while Fortran has multiple standards. So tooling switches are not quite as big an issue.
Lifecycle of languages is a hard issue regardless of language. Unless you keep switching to the correct new shiny thing then you will have developer issues. I think their are a few more Fortran programmers than Delphi these days.
How is Python even being mentioned as a replacement for Fortran? That is like replacing C code with Python just because ...
Sorry I really love Python but I see it as a slow weighing beast that is always a good second best choice for everything it does. Great if you only know Python or only want to know Python but for production? Speed issues is the main reason why Python is not more widely used.
> How is Python even being mentioned as a replacement for Fortran?
Its probably being mentioned as such because a lot of the reason people use Fortran is because of key high-performance libraries for particular numeric application domains -- not because Fortran is necessarily the best language, aside from the existence of the those libraries, to express higher-level solutions in those domains. And because Python has convenient wrappers for those libraries, such that where that is the motivation for using Fortran, Python is a reasonable alternative.
> Speed issues is the main reason why Python is not more widely used.
Sure, and that's why you wouldn't (with the current implementations available) want to write the low-level numeric routines underlying NumPy in Python.
OTOH, once you have NumPy, a lot of time Python makes perfect sense as the language to use for higher-level solutions that rely on the functionality provided by those libraries.
The slowdown from using Numpy (or Octave or whatever) instead of Fortran can be anywhere from 100× to 1.0000000001×. It depends almost entirely on how much of your compute time is spent inside standard computations on large arrays. If your arrays are tiny, or you're doing irregular things to them and can't figure out how to vectorize them, figure 100×. But there are lots of Numpy programs that fire off a few dozen Numpy calls per second, so the bulk of the code executed is Fortran (or maybe C) in any case. The extra microsecond to interpret the Python bytecodes to fire off the call just isn't significant.
More typical is about 3× or 5×, due to iterating over the data many times (once per Numpy call) instead of once, thus bottlenecking on main-memory bandwidth.
Personally, I like to learn new languages, so I might be fairly biased.
But about 18 months ago, I took over maintenance of a small-ish Delphi application when the original developer left the company. It's not a super-fun language for me, but to me it was fairly easy to get into.
I think a lot of people assume Fortran == FORTRAN 77, which is a terrible language without even full support for structured programming and with an annoying fixed-field syntax. There are good reasons to hate on it.
Most Fortran hate is actually F77 hate.
Fortran 90, on the other hand, is basically C with a different syntax. F90 (and newer: F95, F03, F08) really isn't a worse language than C, and there's no reason to hate on it when you wouldn't hate on C, but a lot of people don't know it exists because they equate Fortran with F77.
It seems like you're purposely misunderstanding the point of the article to fit your worldview. This is less about about the new hip things the kids want and more about running a business.
Do you know FORTRAN? Do you know any programmers who know FORTRAN? Do you know anyone who wants to learn FORTRAN? Unless you're in the community the answer to all of those is no. And I can't imagine it's a big or growing community. The obvious consequence of this is FORTRAN will become more and more expensive to run a business on as time goes by because fewer and fewer people will have experience with it or any desire to use it. It's going to be harder to find people to port it and even harder to find people to maintain it.
As to it becoming "more and more expensive to run it", that kind of depends. Most of that Fortran code isn't touched very often. The cost of swapping it out is huge.
Swapping it out piecemeal isn't as expensive money-wise, but it carries large risk. Should it be ported out? Sure. Can it be, in any reasonable way? That's a harder question, and it's not as cut-and-dry as you make it sound.
> Unless you're in the community the answer to all of those is no.
Fortran is still widely used in mathematics and science. So if that's the community bubble you're referring to, then yes, there's definitely a niche there in my experience (but not purely for legacy reasons).
I've got a couple of friends who work at pretty big SAS shops. Non of them knew SAS before being hired. Their employers didn't care about their (lack of) SAS skills and no SAS questions where asked at the interview. They simply hired them and sent them on a couple of 3-4 day SAS training courses and expected them to pick it up.
Are they statisticians by any chance? Their domain knowledge would be much more important than programming experience. Conversely, for a programmer experienced in a classical PL (FORTRAN, LISP, C, Java), SAS is quite a shock. Data-oriented computation models (like SAS has) are not so common any more.
Not really I write a lot of SAS (Engineer at an Industrial plant) it is what we use to query all our production databases and to write Reports for operators.
If you know SQL or anything about databases SAS is very easy to pick up. My background from University was Java and C I had few difficulties.
My House-mate is a statistician he works in finance and uses R my understanding is R is a bit more difficult as it doesn't have anything like SAS's Proc SQL never used it myself though.
Coincidently Fortran is used very heavily in models here as well.
There are idiomatic ways to use every language. Much in the same way that you can write C++ that looks a lot like C, you can write FORTRAN that looks almost exactly like C. So if you have a lot of FORTRAN and thousands of C programmers, it isn't a huge problem. If your codebase was in COBOL, maybe you'd have a problem...
It's a pretty great thing viewed through another lens, you hire kids and teach them FORTRAN. They won't jump ship as easily when their work experience is all FORTRAN.
The problem is that the code base is byzantine with developers hired who don't have good Fortran knowledge.
There are a million different versions of the same function each with a minor tweak to get the "right" behaviors labeled func_name_1 down to func_name_1000000 because everyone is too scared to fix the original function as they don't know what depends on it.
It's not that Fortran is the right tool for the job it's that it has terminal momentum.
> The problem is that the code base is byzantine with developers hired who don't have good Fortran knowledge.
> There are a million different versions of the same function each with a minor tweak to get the "right" behaviors labeled func_name_1 down to func_name_1000000 because everyone is too scared to fix the original function as they don't know what depends on it.
Sounds like a problem with development practices, documentation, and overall code lifecycle management -- and also employee management in terms of skill development priorities.
Those issues aren't particularly tied to implementation language.
It also depends very much on which Fortran we're talking about. Fortran has evolved more than just about any other language out there, and Fortran77 (not a very good language) is very different from Fortran08 (a pretty great language).
Exactly right. I first coded in Fortran in the mid 80s on a mech eng code base written in Fortran 66. Fortran 77 was avoided because it was too new. In the early 90s I worked in reservoir engineering where we used 77. Fortran was the choice for number crunching, especially on hardware like Crays that used vector processors that could be exploited by specialised compilers. Those early Fortrans had very little in the way of abstraction. Data was passed around in 'common blocks', and there was no dynamic memory allocation. Remember that Bloomberg started in the 80s. So it's likely that they have a huge Fortran codebase in one of these old variants of the language.
Most people will point out the support issue, namely getting knowledgeable people to work on it might be difficult. In the case of the code I have worked on while we may rewrite a working system in a newer language it is more typical to wrap known bullet proof functions in layers to present it in new formats; web/mobile/etc.
Most people who know C can pick up Fortran in a few days. I am not a Fortran guru, but have learned enough to write a lot of code for a few scientific projects in school. I really don't understand the mindset that you "need Fortran programers to write Fortran".
Fortran code isn't inherently technical debt, any more than code in any other language is.
> Do you still have a VCR? Isn't it a bad idea to assume VHS is a good medium to archive?
I fail to see the relevance of the analogy; its not exactly as if there is a shortage of -- F/OSS and commercial -- Fortran toolchains maintained and available.
Keeping Fortran code is technical debt in a similar way that keeping your VHS videos is technical debt.
Sure, you may be able to find tools to use Fortran in the near term. But how do you protect the future. Does it play well with emerging systems/protocols? Do you have developers to maintain it? Is there a clear path to upgrade to newer platforms and frameworks?
Two reasons are maintenance and hiring. How many talented Fortran developers are out there now (and living in NYC) available to fix and maintain code that your entire business depends on?
My own take on that was: "Wow, they surely must do a lot of performance sensitive stuff to use the fastest language available for number manipulation. Kudos to them."
I'd argue two points here: Recruiting and technical debt.
There may be challenges getting the talent you want with legacy systems and I believe you're building a technical debt as you'll hit a wall with your infrastructure and have to upgrade/migrate one day anyway.
The top comments on this posting are ex-employees talking about how they've been migrating many (most?) parts of the system to C++, so yeah, even Bloomberg management agrees staying on FORTRAN is probably a bad idea.
