I recently had lunch with Lutz Mueller, creator and author of the programming language newLISP.
I’ve never used Lisp, but I’ve always had a fascination with the “AI” languages. There’s something magical about a language designed to solve ridiculously difficult problems.
In the same vein, there’s something magical about an individual who writes a programming language on his own. So someone who writes an AI language on his own ranks pretty high on the list of intelligent people I’ve met.
Like a renaissance man, Lutz’s experience runs far and wide: he has a PhD in Psychology, a background in statistics, and spent many nights programming punch cards (and many nights working for an internet startup). He’s now the one-man show behind the open source scripting version of Lisp called newLISP.
Rob: Give us some background on newLISP. How long has it been around, how has it evolved, etc…
Lutz: newLISP started on SunBSD in the early 90’s, then moved to Windows 3.0/1 as a 16-bit application, then to Windows 95/98 as a 32-bit program. During all that time newLISP was more like a hobby for me. Around 1999/2000 I made it Open-Source under the GPL license on Linux. At the end of 2002 I started working on newLISP full time and a community of users has been growing quickly since then. In 2005 it was adopted for the first time by an internet startup as the only development language for a large scale (~100 CPU’s) distributed application running a specialized (sentence) search engine.
Rob: How is newLISP different from LISP?
Lutz: newLISP is a scripting language very much in the culture of Perl or Python; quick to learn and to get the job done. newLISP takes its core elements from LISP, such as lists and lambda expressions, and adds name-spaces and implicit indexing.
The style of programming has changed dramatically during the 90s and the internet age. Programming is more casual, interactive, and performed by different people than before the internet and personal computer, and is now used for different sorts of applications. Scripting languages have made access to programming easier for people with no previous programming experience and has greatly increased speed of development for professionals.
Rob: What inspired you to create newLISP?
Lutz: I was always intrigued by LISP but found that the current standards of Common LISP and Scheme were behind modern times in their approach to programming, and too overloaded with academic concepts to be useful for solving real world problems. I wanted to create a LISP with a scripting feel, like Perl or Python, but still a LISP.
Rob: Creating your own language seems like a massive undertaking. How is writing a language different from other software development you’ve done?
Lutz: What you are creating has the potential to be used by many different people with different backgrounds, motivations, and on different applications. Making a language has as much to do with Psychology as it has with Software Engineering. It is very important to listen to users and to know many other programming languages: from machine assembly language to high level languages like SQL.
The core of a well designed programming language should be small. The bulk of the newLISP code consists of the newLISP API functions that are, internally, relatively independent from each other. But looking from the outside, from an API-level, those functions have to work together and should paint a cohesive picture of the language.
Designing and writing the software is only about 25% of the entire project. The rest is QA (Quality Assurance), documentation, and contact with the community of users via support and exchanging newLISP code and ideas. The way a programming language is used differs very much from person to person and from application to application.
Rob: Are you the sole developer on the project?
Lutz: For the core of the language I am the only developer, but many bits and pieces in the project come from other projects and programmers, as is typical for many Open-Source projects. I especially depend on help on the documentation side since I am not a native English speaker.
Rob: Most of my readers are enterprise developers using .NET, Java, or PHP. Why would an enterprise developer want to learn newLISP?
Lutz: Many people like the functional style of programming, but they feel that Common LISP and Scheme do not offer the API they have come to expect from .NET, Java, or PHP. Things like regular expressions, network functions and support for popular protocols like XML or HTTP are all built in to newLISP without the need of external libraries.
Many types of problems, i.e. complex statistical analytics, modeling and simulation are much easier to design and implement in a functional programming style using readily available functions in newLISP. This also includes financial functions and Bayesian modeling implemented on a high level.
newLISP has a documentation facility similar to JavaDoc, which makes it easier to use in an enterprise setting. It also allows splitting large programs into independent modules. But using newLISP is not an all-or-nothing decision, it can live and cooperate well with other languages in a project.
Rob: How does newLISP compare to Python in terms of AI programming?
Lutz: newLISP has better functional abstraction and symbol handling and is more flexible in modeling. The same list syntax for program and data makes it easier to write programs which generate or modify other programs. newLISP has built-in support for distributed applications.
For example, you can use the same function to read/write from/to the local disk or over the network, or the same functions to load program code locally or remotely. A built in function called net-eval exists to distribute processing to many remote nodes in one function call.
Rob: I’ve never used LISP or newLISP. If I want to learn newLISP, where should I begin?
Lutz: On http://newlisp.org you will find introductions created by professional writers and a few small tutorial movies. The newLISP manual and reference is full of examples and there is a ‘Code Patterns’ document describing common solutions for common problems. Traditional books on Common LISP or Scheme are not recommended for learning newLISP, because they emphasize a style of programming not suited for newLISP.
Here are some pointers to pages on the newlisp.org site:
For introductory and reference documentation:
Short code snippets:
User contributed short examples:
Support and community:
A blog about newLISP:
The last link also contains links to other newLISP related sites.
Rob: Have you seen the Netflix contest (www.netflixprize.com)? Is newLISP up to the challenge?
Lutz: I believe this is a typical problem that can be solved with newLISP, but I don’t have the time and do not quite agree with their terms. Perhaps someone else will take on the challenge using newLISP, it would be the best tool for this contest 😉
Rob: Thanks for your time, Lutz.
Lutz: Thanks for talking with me.