Shah, who grew up in Mumbai and was among those who helped architect Aadhaar, created Julia about 10 years ago together with three other technologists – Alan Edelman, Jeff Bezanson, and Stefan Karpinski. He’s also now the co-founder and CEO of Julia Computing.
Shah was our guest on Times Techies webinars last week and he spoke to us from Boston where he lives now. Julia is an open-source language for high-performance technical computing and data science. Shah said Julia is over 10x faster than Python and R, both of which are also primarily used for analytics and AI. Asked how they did it and whether Python could be made equally fast, Shah said: “Can I take my Tata Indica and bolt on a new engine, change the fuel and with customisations, can it become a Ferrari? The answer is no. Ferrari was designed to be a fast car and every decision was taken keeping performance in mind. When Julia was designed, every design decision from day one was taken keeping in mind the performance. If it didn’t help performance, we didn’t put that feature into the language. We have to start from the drawing board and not retroactively fit it into an existing system.”
Python is a general purpose language and is being used increasingly in areas other than data analytics and data science. But Julia is focused on big data and analytics. “If you’re building a new search engine that’s heavily mathematical, or trying to predict the weather, or discovering a new drug, that’s where you use Julia. But if you are building a mobile app, Julia is not an answer to that. It’s for very large data sets and where you are building a complex algorithmic application,” Shah said.
Many Julia libraries are written by students, many of them from India. Half of the Julia community participants in the Google Summer of Code come from Indian universities, mostly from tier-2 colleges. That talent is amazing. Things that are written by PhDs and professors are being written by students in India. They have raised the bar.
Viral Shah, co-founder & CEO, Julia Computing
He said Julia is also excellent for the production environment. Scientists and domain experts use Python, R and SAS to write their algorithms because they are simpler languages. But when these algorithms have to be run on data on large scale – put into production, in other words – the algorithms need to be rewritten in C++, C# or Java. So two different languages, two different sets of people required. Shah said Julia can be directly put into production, and that also allows the scientists, domain experts, data scientists and programmers to work on the same platform, leading to a lot more collaboration. “Julia is as much of a social construct as it is a technical construct,” he said.
Julia, he said, is enabling the ability to do differentiable programming – what makes the learning in AI possible – on a broad variety of systems compared to AI algorithms today. This makes it possible to use AI in many new areas.
Asked if mathematics was necessary to use Julia, Shah said it wasn’t necessary. But he said mathematics will make one a better programmer. “Julia is the language of science and the ease with which you can compute using math in Julia is far superior to what existed before it,” he said.
Julia is being used by Google as a research language. Shah noted that Google’s head of AI, Jeff Dean, tweeted two years ago that Julia in combination with Google’s hardware – Tensor Processing Units – was the best combination for fast and easily expressible machine learning computations.
Intel, Amazon, Facebook, Microsoft, Uber are among over 10,000 companies that today use Julia.