Transcript: Emacs chat with John Wiegley

This post is long, so if you’re reading this on the main page, go to to view the full transcript! Here’s the video. You can also download the MP3. John Wiegley – June 26, 2012 from Sacha Chua on Vimeo . A day in the life of John Wiegley Sacha: One of the things I’ve always been curious about is all the different things that you use Emacs for. You’ve been one of our role models for ages now, and clearly you do a lot of Emacs Lisp programming with it, but what’s a day in the life of John Wiegley like? J: I spend the most of my time in Org and Gnus. All of my task management… I think I’ve processed over 5,000 tasks in Org mode now, since I started using it. I’m a very very heavy Org mode user. I’m always in Gnus, always checking my e-mail through that. I use ERC. I actually run a second Emacs. For my Mac, I built another Emacs under another name, and I use that Emacs just for running ERC. I use that in conjunction with bitlbee so that I’m always on IM, always on IRC, and also that’s my Twitter client as well. So that’s always running on the side as well. I spend a lot of time then in Eshell, all the programming modes… Most of my day work is in C and C++ when I’m not hacking elisp. S: Why do you keep your ERC in a separate Emacs? To minimize distraction, or… J: When I’m hacking on Emacs, I end up needing to restart it quite often. Many many times a day, sometimes. That’s because I never know which definitions… Sometimes you change a definition from a function to a macro or vice versa, and you don’t know which other definitions you have to reevaluate in order for them to inline the new definition. Rather than have to figure that out all the time, I just restart Emacs. S: Hence your trick of making sure that everything’s compiled and that you’re requiring all the files you need so that it loads up cleanly. J: I just recently fixed a problem in my .emacs and I discovered that compiling it was not giving me any speed benefit. I thought compilation was what was making my .emacs run so fast, and it wasn’t. It was that I was loading—when I was running a non-byte-compiled Emacs,I was loading things that I didn’t need to load. When I fixed that problem, which is now fixed in my .emacs repository, Emacs loads in just over a second, but without doing any byte-compilation in my Emacs. S: I must definitely be doing something long, because my Emacs takes a while to load. J: How long? S: I don’t know. I tried using the profiling thing, and because I use the Emacs Starter Kit, it didn’t get very deep. It feels like ten seconds or so. It takes a while. I can’t be bouncing it up and down like you do. S: So you do a lot of Emacs Lisp programming, naturally. You’re on ERC, and you’re doing C and C++ development. Are there other really weird things you do that people wouldn’t expect Emacs to handle? J: Let me think… I use it to play chess online. S: Yeah, there are so many games in there! I play Nethack in there, so… pot, kettle, black, here. J: I use it to look at databases. I use it… Let’s see… I of course use TRAMP to edit not only files remotely, but also local files through sudo so that I can edit them. Weird things I do in Emacs… A mode I forgot to mention is that I use git for all the version control that I do, and so magit is a mode that I just basically live in. For any project that I’m working on, the magit buffer becomes the home buffer for that project, and I’m constantly looking at that buffer to see what work I’ve done, what work should be committed now. S: I haven’t made that a big part of my workflow yet, but I’ve heard such good things about it. J: It’s a very nice mode. I use it in conjunction with the built-in vc mode of Emacs. If I’m editing a file and I really quickly want to know what I’ve done to this file, I’ll do C-x v = to get the diff of the current file. If I want an overview of what have I been doing and what I have been touching, I’ll go to the magit buffer and look at the stats. S: I guess you version-control your Org files too. How many Org files do you typically work with, and how do you manage that? J: I have eight. All of my active tasks exist in a single Org file. The other seven Org files are all archives. I have an archive file for every project, even though the live project lives in the main TODO file. That way, when I do an Org search, it’s only at that time that it loads in all of the Org files to do the search. I need Org to be as quick as it can, since I’m basically modifying tasks and adding tasks to it as the days go on. S: That could be it. I’ve got a humongous Org file and org-refile takes a while trying to parse it. J: Yes, it does, it does. I use Dropbox to synchronize my Org-mode files to my iPhone because I use org-mobile. Another interesting tool I have found is that there’s an app for the iPhone called dropVox, which lets you take a voice note and it puts it into Dropbox. I have an Org-mode hook so that whenever I open my Org-mode buffer, if there’s an audio file in my Dropvox box, it will pop up Dired and show it to me, meaning “You should listen to this and add it as a note to your Org mode now.” S: That’s awesome. That’s like If This Then That on steroids. J: All during the day, new tasks are coming in. They’re either coming in by ideas, by e-mail, by webpages… I have a keybinding I use in Emacs. Since I don’t use M-m for anything else, M-m is my make-a-note keybinding. So, wherever I’m at, if I hit M-m, it will make an org-capture and it will link it to whatever I was on when I did the capture, and it will link it to whatever I was on when I did the capture. If I was on an e-mail, it links it to the e-mail. Well, there’s a tool for the Mac called Quick Keys, and Quick Keys lets you rebind things globally on your system. I’ve made it so that M-n works anywhere on the system and tries to be as intelligent as it can. If I’m looking at Chrome and I hit M-n, it will take me to Emacs, pop up the org-capture buffer, and then put a link over to the webpage that I was looking at. S: Is that using org-protocol or just QuickKeys? J: It’s not using org-protocol. It’s just using QuickKeys. The only thing QuickKeys is really doing is it’s switching over, and then I use AppleScript from Emacs to talk to Chrome and get the information. I actually use AppleScript quite a lot for many different things. Using AppleScript from Emacs is something I do often. S: What are some of the other AppleScripty things you do with Emacs? J: I don’t like to keep Dropbox running all the time because it takes lots of background CPU. At the end of a few days, when I look at my process list and I look at total time in the kernel spent by all the process, Dropbox is usually behind the kernel itself. That’s a little egregious to me when I’m only using it once in a while. I have Applescript so that in Org Mode, when I say go get my mobile tasks, it starts up Dropbox, waits half a minute, and then stops Dropbox. So it’s just running enough time to do this synchronization. And of course, I use that async module that I told you about last week to do that work. S: It sounds like you’ve got it quite integrated into the other things you use on your Mac. That’s fantastic. J: Emacs is the center of my entire environment. S: Being able to glue all these bits together and make things work… That’s incredible. On wishlists and Emacs Lisp S: What are some of things you wish you could glue together? What’s the John Wiegley to-code-some-point-in-the-future list? J: I’d like it if Emacs had a foreign function interface so that I could talk directly to databases and other things. There’s a fork of XEmacs that can communicate directly with PostgreSQL, and something like that would be nice because there are some systems that I work with where it would just be faster and more efficient if Emacs could talk to those systems directly instead of me having to communicate with them over a process. Embedding a Python interpreter, or embedding a Ruby interpreter… S: Vim is extensible in a couple of different languages now, isn’t it? J: I do prefer Emacs Lisp. I have to tell you that probably of all the languages I’ve used, definitely, Emacs Lisp has been the most fun. I won’t call it the best language out there because it does have its downsides and it’s a little bit slow. I can’t use it for most general tasks. But it’s fun, because you see results immediately, the debugger’s easy to use, the documentation is great and completely available at the tip of your fingers. It may be true that I have written more new code in Emacs Lisp than in any other language by this time. S: I wouldn’t be surprised. J: I’ve worked on much bigger projects in C and C++, but those didn’t always involve spitting out reams and reams of new code. Whereas as the day goes by, I’m writing new Emacs Lisp functions usually left and right to get particular jobs done. S: I’m always running into your name. Oh, Planner! Oh, remember! Oh, eshell! J: Too bad not all of those projects succeeded as well as I’d hoped. S: Going back to talking about org-capture and picking up an annotation really quickly… I remember when we were playing around with that. Finding ways to hook parts of Emacs into all the different parts of Emacs… It’s great to see so many people playing around with these ideas. The Emacs community J: It’s a great community. It’s a good culture around Emacs. S: How did we end up with something as cool as this? Emacs is pretty unique among the software packages or open source tools I’ve seen. Vim users are pretty happy and they share a lot of tips, and on the other end of the spectrum there’s Eclipse, and there’s a ton of development work around Eclipse, but Emacs… it’s old, but lots of stuff is going on. Why? J: My opinion on that would be is that the real success was the Lisp machine. The Lisp machine was an entire machine that was what Emacs is to editing. You sit down at your operating system and it doesn’t matter what you’re using – your editor, your shell, your document viewer, whatever – they’re all written in Lisp. You can modify them as you go. The documentation for anything is available as you’re looking at it. You can pop the system into a debugger at any time. Lisp machines may not have succeeded, but Emacs Lisp… Emacs took that environment and that idea, and brought it down to the domain of a single application, the editor. It gives us all the cool things about the Lisp machine: the fact that the debugger’s available all the time and the documentation is completely cross-linked with everything. I think that’s probably what we’re experiencing, and that’s why it’s so much fun. S: … And the fact that you can get in, you can tweak just that little thing just a little bit, and eventually end up with this massive Emacs configuration because you’ve been tweaking it to fit you. J: I have to say that the original designers and Dr. Stallman – they had a very good idea when they put in a lot of hooks throughout Emacs. There are other extensible systems out there in the world that are not as extensible because they lack sufficient degree of hookage inside, places where you can latch on a piece of code to execute when something happens. Emacs has got those everywhere. That plus its advising system lets you basically change the behaviour of anything or augment the behaviour of anything. S: I have to confess it’s one of the things I like the way that Ruby on Rails will let you open up classes, redefine functions, and then continue on with your work. The extensibility built into the very language is very very helpful. It also can be very intimidating. Learning Emacs S: We’ve talked about this before. You’re maybe one of a few Emacs users over there. I’ll on occasion run into someone who’s curious about Emacs but hasn’t taken the plunge. How do we get more people interested in this stuff? J: Well, getting them interested is not that hard. It’s getting them to climb the learning curve that’s difficult. My wife’s a physician, and she sees what I do with Org-mode. She’s been tempted to learn Emacs just to use Org-mode. S: I hear a lot of stories like that. J: But the learning curve is so enormous that she just doesn’t have the time to learn it. S: At some point you were very much into Vi, and then you said, okay, we’re going to learn things the Emacs way. You just sat down and you did it. Is that something we expect people to sit down and do at some point? Have you come across any things that make it easier for people? J: Not necessarily that make it easier for people, unfortunately. I think it’s a philosophy thing. I use Emacs. I’m in Emacs and I use Emacs probably 70% of my every working day. It pays dividends to master it. Every efficiency gain I get in Emacs, I get to make use of right away, and it pays off as the days go by. There are people who type for their living who don’t know how to touch-type. That, to me, is the exact same scenario. How can you make your living as an engineer, typing day in, day out, and yet lose the productivity that you would gain by learning to touch-type. Even learning to touch-type – yeah, it will take you a few weeks. You either use a piece of software or go to class, whatever. So there is a hump that you have to get over, and you may not have the time to get over that hump right now, but it’s an investment, and that investment will pay off. S: Get to know your tools and get to know them really well, because you’re using them all the time. In terms of Emacs, Emacs being very very big and Emacs being something that moves very quickly, what are some of the things that you want to dig into and learn more about? J: I would like to learn the C side of Emacs more. I’ve never known the C side of Emacs. I’ve just recently been looking at the bytecode interpreter and trying to learn how it does what it does to see if there are ways to get better performance into Emacs. That, for me, is the undiscovered country. That’s where I want to go next. S: It does sound like a lot of deep magic. That’s the very core of it. J: It’s not as crazy as it seems. It’s pretty well done on the inside. Emacs without all of its Lisp modes and packages on top of it, if you boil it down to just its essence… the kernel is not really that huge. It’s a very small, very tidy, simple thing. Of course, there are places where it has some rough edges that can be smoothed, but it’s not what people think of as Emacs. They think of this kitchen sink application that does absolutely everything. That’s a lot of Lisp stuff that goes around the little kernel, whereas the kernel is very tight and small. I want to know more about that because anything done in the kernel affects everything else. S: If you ever get around to doing annotated source code, I’d definitely read that. Making money with Emacs S: I hear you’re kinda on the hook for eshell documentation or whatever else people would like you to write. J: That’s true. There’s a reason why the eshell documentation was never written. This would be a whole different discussion. I have some misgivings about what kind of world the GPL would create if it was everywhere. I do a lot of my programming as a hobbyist, but I have to make money programming as well. The way to make money through software is usually to sell it. Otherwise, if you make money only through services, that never takes off. If you make a piece of software and you license it, it can take off. It can start making money for you and you don’t have to work to earn every day. Then you can use that time that you now have to make more software. If the only income that you ever made was based on services, then you basically have to be working all day long, and when would you ever get your hobby coding done? When you only have six to eight hours a day to do any coding at all (because there are things that we have to do), you want to have a setup where you can do as much creative coding as possible. Since the GPL’s view of the world is that you get paid through the services and you get paid through the documentation, when I released eshell, my thought was, “Okay, I’ve written the code, the code is in the GPL, so it’s freely distributable and I can’t charge anyone for it, but if they want services around eshell, then they can pay me for that.” I have always told the community that if someone wants to step up and pay for it, I’ll write the eshell documentation. But that’s never happened. So if the community doesn’t value the eshell documentation enough to pay me to do it, then why would I spend the time that I could be spending coding to write it? S: Do you know what kind of bounty system we have or something like that for lots of people to say, “I want to pitch in so and so much to e-shell documentation?” Do we have that? J: Yeah, or a Kickstarter project, for example. S: That would be cool. New users and Emacs happiness S: You’ve been an awesomely prolific Emacs Lisp programmer, so it would be interesting… J: Well, it’s just that I’ve been doing it for so long. It’s been eighteen years now since the first package that I wrote and submitted to Emacs. I know! You were just a kid back then when I was writing align.el. S: I was ten! I’ve used align.el. J: Yup. It was made when you were ten. S: Are you seeing a lot of other young people get interested in this stuff? J: Sure! It’s basically if you’re not going to be using an IDE like Visual Studio or Eclipse or something, Emacs is still one of the two great editors out there. It’s either going to be you go with Emacs or you go with Vim. It still pulls in new people all the time. S: There’s just so much. Once people start customizing it, they get sucked in. As you said earlier, there’s a lot of interest in Emacs from the nontechnical side of the world. Writing, scientific papers… J: We’re getting a lot of new users just because of Org mode. S: How many years ago was that? Now, it’s just grown into this massive thing where people are writing their research papers and they’re doing their data analyses in org-babel, and having something come out… Literate programming writ large. J: I started using it in 2007 and I think it was a couple of years old by then already. I tried to drop Org-mode a couple of times. I was thinking, there are sexier-looking apps for the Mac. There are apps that have better and tighter integration for the iPhone. On two different occasions, I left, converted all of my tasks over to a different program, used that program for a few months, came back to Org… and I always felt happy to be back in Org. I don’t know what it is about it. It looks right. It feels right. It’s got the right balance between how finely you can enter and manipulate the information, and how coarsely you can look at it at a glance. Other applications that I used… I don’t know. There was just something about them, but I wasn’t getting the tasks done. I would put all the tasks into the application and I’d be excited about it for a few weeks, and then after a couple of months, I just wouldn’t look at it any more. I would know that the tasks were in it, but I would never do anything about them. The way I use Org Mode, I use it like a day planner, so that every task I intend to do is scheduled for a particular day. I’m rescheduling tasks and moving them to new days every single day for years now, and it just never has felt like a burden. So there’s something that Org does right. S: There’s the hack that you told me about the other time where you change your window size, so you watch it shrink as you finish your tasks. J: I fit it to the window. S: Little motivational hacks that you can do because you can play around with the tool itself. I remember when I was trying to learn through flashcards using flashcard.el, I rigged it up so that it would tell me a joke using fortune.el and everything I got something right. It was either that or show me a cute cat picture from the files I’d saved off icanhascheezburger. The fact that you can hack it to do all sorts of crazy things… that’s incredible. J: I just started playing around with its ability to view PDF files. You can use C-x C-f to open a PDF file and you’ll see it in your Emacs. It renders them page by page as PNG files, and then uses the Imagemagick extension built into Emacs 24 to show you those pages. Which is useful to me because I’m often looking at a language specification as I’m writing code, and it’s nice to have it in another buffer the way that I would have Emacs documentation. I can look at the C++ standard now and have it just be in another buffer. S: How did you come across this new capability, because I didn’t know about this? J: I think I ran into it accidentally. I think I was in Dired mode and I hit return on a PDF instead of hitting ! to open it, and all of a sudden, there it was, and I was, like, “Wow, I didn’t know Emacs could do that…” S: Basically, for people who want to learn things… Just do random things in Emacs! J: Although if you’re going to do random things in Emacs, take notes. Otherwise, you’ll never know how to get back to what you found. S: That’s what the lossage buffer is for, isn’t it? J: The lossage buffer can be a bit hard to read, though. S: Execute-extended-command? J: I did find on Emacswiki a mode called Command Log. It keeps in a very readable form every command you use. S: I definitely have to pick up this habit of yours of just reading Emacswiki. J: That’s how I started learning. You asked about how people get over the learning hump. I’ll tell you what I did. Back in 1994, when I started really wanting to know Emacs, what I did was I printed out the Emacs manual, which at the time was I think seven or eight hundred pages. It was just single-sided paper. I probably killed a small tree doing it. But I brought that stack of papers over to my desk and I put it to the side. At the time, my machine was slow enough that I was often waiting for builds to finish. What I would do is while whatever was building on my machine, I would pick up the top page of the Emacs manual, I’d read it, and then I would throw it away. I just did that over several weeks’ time. I ended up reading the Emacs manual in all of this dead time that I had, waiting for compilations to finish. I made that a yearly habit for the first four years, just to constantly refresh my knowledge of what’s in there, because it’s such a massively huge environment. It helped. S: My story is that I’ve used Emacspeak to synthesize the Emacs manual so that I can listen to it while walking around. I was reading my mail off Gnus at the time. You could use Emacspeak to read your mail and all of that stuff. You find all these ways to cram information into your brain. I would be up for more podcasts. I see people are coming out with books as well. There’s the Org mode book. That might be another way for you to do it, right? You write your documentation and you say, here’s a book that you can buy. But then it’s very speculative work, I suppose. Back to earning S: Speaking of other things that integrate into Emacs… Thank you for writing Ledger, by the way, because I still run all my finances with it. I have no idea where I’m going to find an accountant who understands Ledger… J: Our biggest problem right now. S: Either we put together a Kickstarter so that you end up writing a manual and accountants all over the world will be like, “This is awesome!”. Or I just learn how to use Quickbooks. J: Ledger does have a manual. That was one I wrote the manual for. It’s also not GPLed. S: There you go. There are ways to work around that… So your ideal is figure out how people can pay you for documentation because all the code is GPLed anyway. Are there other models that seem to be working for other people, other ways to make the awesome hackers that work on this stuff happy so that people can keep working on this cool stuff? J: Not with Emacs hacking. I’ve been paid one time now to do a course on Emacs because my company does training, and Emacs training is one of things that we do offer. S: You should definitely explore that remote training aspect. J: We’re looking into that. I believe that is the only time in my life that I have earned money just because of Emacs. So it hasn’t paid for itself monetarily, but it’s paid for itself in other ways. S: In terms of efficiency, being able to do all these things and fly through that. Anyway, that might be an interesting challenge for us also, figuring out how we can get more Emacs geeks to be rich and famous. J: That’ll be the day. S: Imagine if we had an Emacs app marketplace? J: Yeah, seriously… Wow. Just propose that idea on the mailing list and see what a flamewar that would begin. The developer community S: But it is very nice to be able to play around with all these packages, and there are thousands… A lot of them will go and look for ways to integrate with each other, like the way BBDB integrates with Gnus. J: That’s another one that I use. And there’s always new stuff coming out, and authors are very good about interacting with each other. The author of helm just recently incorporated using my async module that I wrote. He did that just in a matter of a few days. Since helm is something I rely upon on all the time, I’m very happy to see that. S: That’s actually one of the challenges I came across when it came to writing documentation or writing a book about Emacs, especially the modules that people are working on. The packages, right? You share an idea, and the maintainers would be, like, “That’s an excellent idea!” and they would fold it in. So I kept running out of book topics! That’s a good thing about the community. It moves so fast. J: If people are looking to know Emacs better, they should also stop by the IRC channel. I’m there every day, and a lot of people there can give good help if you have questions. S: Yeah. It only looks off-topic from time to time, but if people show up with Emacs questions… J: It also depends on the time of day, too. S: Do evenings tend to work for you? When do people usually hang out there? J: I’m a night coder, so that’s when I’m there, but people are there all around the clock. S: I tend to drop by in the evenings too, when I remember to do that. ERC makes it so easy because it’s just there. J: Yeah, I haven’t seen you there that often. You should come by more. S: I know. J: I always see you at the very end of my night, what would be dawn for most people. That’s when you usually come in. S: That’s funny. I’ve got to work on my timing. One of the things I want to do is figure out if we can have a regular Google Hangout or whatever, right… It’s easy to do screen-sharing through that and you can have multiple people, so, if we just get people together and say, “What have you learned about Emacs lately?” then it’s slightly more visual than IRC. J: There’s also Twitter. A lot of people use that to talk about new stuff they’ve found in Emacs. S: Yeah, I’ve seen a couple of people tweeting really short Emacs tips, and it’s great to see that kind of stuff going on. I remember people used to have Emacs tips in their e-mail signatures as well. All these little ways to increase randomness. Emacs is so huge that if you just find little ways to say, “Oh, hey, there’s this new feature” or “There’s this interesting command over here,” who knows what it’ll spark? J: There’s too much good stuff. More tips for new people S: So you recommend that if people are new, they should check out the IRC channel, EmacsWiki… J: Go through the Emacs tutorial first. Then stop by the channel, read the Emacs manual, pick something you want to accomplish with Emacs and focus your learning around making that happen, rather than taking on the task of trying to swallow the whole thing right up front. S: People will easily divide into… You want to do programming, check out the EmacsWiki pages on the respective languages. If you want to do text editing or the subspecies of research paper editing or whatever else, then there are pages for that too. J: There’s always more stuff out there than you’re aware of. I’ve been trying to make myself aware of everything that’s out there, and I keep running into new stuff on a daily basis. Just yesterday I found swank-js, which lets you interact with the Javascript interpreter inside your browser as a REPL. You can connect to Firefox and be manipulating the webpage through an Emacs REPL using Javascript. Isn’t that crazy? If I were doing web development, I could see that that would just be invaluable. S: I will go have to check that out. You’ve also mentioned a couple of other Emacs blogs, like Mastering Emacs… Is that one syndicated on Planet Emacsen yet? J: I don’t know. I’m not sure if I have… I aggregate all of my feeds in Gnus into a virtual group, so I’m never aware of the actual source of any feed, I just get presented with one group that has all the current happenings in Emacs. Emacs blogs and reading with Gnus + Gwene S: Someday we should totally get your OPML and cross-reference it with everything that’s syndicated there. J: That’s true. That wouldn’t be a bad project, maintain an OPML file of all the Emacs feeds out there on the Net, because I would love to keep my list updated. I went through and did a search through Gwene to see all the feeds that it had syndicated, which is of course not all the feeds that are out there. I did that two years ago, so there have been new feeds since. S: I’ll take that away as a TODO and see if Planet Emacsen will have all these things… Who was that in charge of it? Edward, Edward O’Connor. J: Yeah, that’s right. So any time you find a new feed, add it to Gwene. That way, anyone who goes to the Gwene server with Gnus can just do a search for all groups matching to Emacs and

You can listen to Transcript: Emacs chat with John Wiegley online on Radio and Podcast. Open the player on this page to stream the available audio.

Transcript: Emacs chat with John Wiegley is an episode from Emacs Chat.

The episode duration depends on the source podcast feed and may not always be available.

This episode was published on Jul 2, 2012.

Yes. Use the heart button on the episode page to add it to your favorite episodes list.

Yes. This page shows related episodes from Emacs Chat when more episodes are available from the podcast feed.