2015-12-10 Emacs Chat: John Wiegley on maintaining Emacs and how you can help

2015-12-25: These are the bugs that have the “easy” keyword. Note that some of them are because of the package or mode name. =) John Wiegley shared how he uses Gnus and Org to help him with the volume of Emacs-related information, and how people can get started with Emacs development. 0:02 Gnus for mail and news 0:04 Organizing groups by topic 0:05 Adaptive scoring and prioritization 0:09 Setup for mail: Gmail, Fetchmail, Dovecot, Gnus 0:11 Time: 1-2 hours a day 0:13 Community-building 0:15 Using Org to keep track of initiatives 0:19 Reading bug reports in Gnus 0:22 How people can help: tests, documentation, reviewing bugs 0:24 Coverage 0:33 Efficiency, benchmarks 0:40 Magit, Projectile, Flycheck 0:45 Following up on emacs-devel topics: IDEs, APIs, lexical binding, Guile, etc. You can e-mail John Wiegley at johnw@gnu.org . The emacs-devel mailing list is at . Event page on Google+ Ogg Vorbis (audio only) MP3 (audio only) View the full blog post for the transcript. Thanks to Phil Hudson for volunteering to transcribe this! s: Hello, everyone! Hi, I’m Sacha Chua, and this is another Emacs Chat. It is December 10, 2015. I’m here with John Wiegley who has, since the last time we talked to him, become the Emacs maintainer, among other things. So we’re here today to chat about how he’s dealing with the increased workload and other cool things! John, over to you. j: Thank you, Sacha. I’m glad to have this opportunity for us to talk again. It’s been a few months. s: By the way, if you have questions, you can submit questions by commenting on the Event page where you found this, ????? etc etc, or using the Q&A app that shows up, or you can pop into IRC on the Freenode network and look in the channel and pose questions there. I’ll try to remember to sneak questions in at the appropriate moment. But I want to kick things off with some questions about your new workflow now that you’re dealing with the increased workload of a high-volume mailing list/newsgroup, called emacs-devel, and other things you deal with as Emacs maintainer. So, what do you do? j: Well, certainly the emacs-devel mailing list is quite high-volume. My technique for approaching that has been to customize gnus until it makes it a more manageable source of information. I’ve had to tweak the scoring setup that I use. Gnus has a feature called “adaptive scoring” where it will observe what you read and what you don’t read and it will try to change the ordering and the presentation of the things you see in the *Summary* buffer when you go in to the newsgroup, based on your reading preferences, by observing your reading habits. That has been extremely helpful. That way, if a conversation has gotten very, very long and I realize that there’s nothing more for me to contribute on there, I can hit a key and never see that conversation ever again, unless somebody uses some sort of keyword that I have a flag for that will pop that thread back up for me. Meanwhile the threads that I’m most interested in, or the ones I’ve been replying to, they will go to the top of the buffers, so they’ll be the ones that I read first upon re-entering the group. s: So you have something that automatically prioritizes the threads for you, based on whether you’ve replied to something or whether you’ve decided this is going way off tangent until it goes back on-topic. Not that I should be asking you which keywords should bring things back to your attention… j: Right now, it’s “John?” (with a question mark): that’s the one that will pop any thread back up. s: Alright, you heard it here, folks, if you need to get his attention, just say “John?” and John Wiegley will appear–himself–to deal with whatever’s going on. Actually, since you can share your screen through this videocast, is that something that you want to quickly demonstrate? Like, how this works. Not sure if that works live, because you may have to read things [out]. j: As I click on the screen-sharing button, it’s not having the [?????] s: Hah! Technology. OK, well, you keep trying that. I’ll try to fill in this space… j: Why would it not be there… s: Hmm. OK, we’ll demonstrate with some hand-waving… OK, actually, one thing you can try–this may not break the system: if you refresh your browser, you might get dropped back into this conversation and that thing might work again. Let’s give that a shot. … OK, so while he’s coming back, and I hope this works, because otherwise: awkward! Gnus is a mail- and news-reader for Emacs… j: Yes, that did work! s: Perfect! All right. I was just telling people what Gnus is. If you would like to go ahead and demonstrate how this helps you with the prioritization… fantastic! j: Are you seeing my Emacs screen? s: I see your screen. j: OK, so, with the key that I have bound, I will jump into Gnus. One thing you see here is that Gnus has the ability to create topics and to place groups under those topics. Though I actually have quite a number of topics, as you can see. s: Wow. I did not expect that. j: Yes. Normally a topic that has no unread groups in it is displayed, and there’s a key-binding that toggles these things. So we see that emacs-devel has gotten two emails since this livecast began. So if I hit return on those… I have the display customized right now so the number on the farthest left is the score that has been assigned to that article, just because I have been trying to fine-tune my adaptive scoring rules, so I want to be aware what effect my actions are having on the scores. I have another key to reveal all the threads. If I go back out and I grab the last 100 messages for example, then I can see that there are some very lengthy threads. I have a number here in this column [highlights fifth column – how do you do that?] that shows me how many messages are hidden underneath the thread. Normally I start and I just see this view, where the thread with the aggregate highest score goes to the top. s: Wow. So this is a thread with a 90 score and 42 messages under it? j: Well, 90 was the score of the top message in the thread, or it might have been the aggregate score. I can hit a key to see what the score of the thread itself was. So 90 was the aggregate score, and the score of the article itself was 30. It would have gotten the score of 30 because it would have noticed me reading a message from this thread in the past. Every time I read a message in the thread, the subject of that thread gets a bump of 1. I must have read 30 messages within this thread already, to have a score of 30. s: So folks, write good subject-line headers so John takes a look at your message. j: Subject lines are important. Gnus is very handy for example when we have a really long thread like this and someone wants to bring up a side point. We instituted a policy of changing the subject line when you do that. When you change the subject line, Gnus has a way of making that appear as if it’s a new thread. You see this “Casting as wide a net as possible” thread? This actually was a part of the “First draft of the Emacs website” thread, but because the poster, John [Yates] here, chose a different subject line, Gnus presents that to me as if it were an independent thread. s: And that also allows you to track your interest through adaptive scoring, for that particular subtopic. j: Right, right. It all plugs into adaptive scoring quite nicely. As a result, even though there are days when the number of unread messages in the emacs-devel group will easily reach 200 by the end of the day, it’s not that difficult to get through them because Gnus presents them to me in roughly the order I ought to read them. If I start seeing that a thread does not really need my involvement, I can start scanning quickly, or just tell gnus I don’t want to participate in this thread from this point onward – unless, of course, someone asks for me. s: You’re getting quite a few people more interested in Gnus, according to the channel. j: Gnus is incredibly wonderful. Let me tell you, Gnus and Org both, they’re my mainstays in Emacs. They’re the two applications that I always have running. Gnus is somewhat like Emacs itself. There is a learning investment to be made, but I do believe it pays itself off, many times over. Further, Lars has done a brilliant job with the gnus manual. It is not only highly informative, it is very funny. It’s worth reading from front to end, more so than any other technical manual I have read. I get quite a kick out of many of the paragraphs in the Gnus manual as I go to read about different things. s: Was it you who also pointed out that the Calc manual was also very interesting reading? So it turns out that the Info manuals for Emacs packages tend to be quite interesting reading and very useful reading too. j: Oh yeah. I would recommend to anyone who wants to get started with Gnus to first acquire a brave heart. It’s not going to be an easy task, but… Go through the setup instructions. Get it connected to your mail source by either reading the mail from a directory or using it over IMAP. You will find that using Gnus over IMAP is very slow. This puts a lot of people off. What I do is, I have all of my emails get to delivered to GMail, because they have excellent spam-filtering services. I use fetchmail to pull the mail from GMail down to my local machine, where it gets queued into a Dovecot server that I run locally. Then Gnus talks to the Dovecot server over IMAP, and that is a very fast connection. And it lets me do local searches of emails, so that even if I don’t have Internet connectivity, I can always search all of the email I have. That’s quite a lot of email. In my dovecot, there’s probably close to a million email messages in there. s: A million! One of the things that I want to mention is that a large part of that million is the entire emacs-devel archive, right? j: Well, it’s the emacs-devel archive, the emacs-bugs archive, the emacs-help archive, the sources archive… Usually, if I actively participate in a mailing list, I like to go download all of its past email, so that if people make reference to previous discussions, I can see those discussions without having necessarily to be online. s: And that takes advantage of Gnus’ ability to show you all of the messages within a given thread. It knows about them. j: Right. Let’s say I’m on this message here, which is a response to this thread called “New update of the Emacs website”, and I want to see the context. I can hit a key, and now all of the members of this part of that thread come into my buffer. So you’ll see the lines of blue here, the ones that are now inserted… I’ll do that again, so you can see what happened. Let’s go in here [emacs-devel summary buffer] So we see that there are three lines in the summary buffer. I hit A T , and now there are 25 lines in that buffer. So I get to see the message I’m reading now in the full context of the thread in which it appeared. s: Right. So Eric has a question, which he submitted through the Q&A thing. This seems like a lot of reading. How many hours a week do you spend on Emacs maintainer stuff? j: I would say that in reading the emacs-devel mailing list, it’s between one to two hours a day, probably (and responding to them). I do not spend as much time fixing bugs or writing code as other contributors to this project do. So for example, Eli Zaretskii spends a very large amount of time addressing bugs and taking care of the stability of Emacs. I rely upon him a lot for his technical expertise. I spend more time reading the messages, trying to see which people issues there are, what things need to be changed in our organization, in our structure, so that all of these resources that we have in the community can be effective and can smoothly head us toward the next release. We have a lot of excellent people, excellent developers and volunteers and contributors. It doesn’t necessarily need me to do all this work. We have excellent people to do that. I just want to make sure they have what they need to get their jobs done. s: That’s an interesting approach to take in terms of maintaining an open-source project. It’s less about making all the technical contributions yourself and more about unlocking the awesomeness in the community, coordinating people’s efforts, reaching out to people and pulling them in. j: Right. And specifically I want to emphasize or focus on that at this point in time, because I feel like we do have a much better community than we’re taking advantage of for core Emacs development. There are the Emacs developers, who at the moment are fewer in number and maybe even a little bit overworked. Then outside of that core community, we have a much larger community of people doing excellent work on external projects that are not part of core Emacs or are not in ELPA. They might be, for example, in the MELPA repository. If I can bring some of those people and attract some of those outside people into the core development, then our job gets a lot easier. That’s going to be much more effective than just me becoming another programmer within the Emacs project. If I can get five other of those people to come in, that’s going to be five times what I could have contributed, for example. s: That’s awesome. So I guess some of these people might be hiding out in emacs-devel, but where are you keeping an eye out for these people you can recruit basically and bring into core? j: In IRC, in Twitter, there are people that I know personally who are for whatever reason on the outskirts of emacs-devel. Apparently–they’ve told me–there have been past issues with things not staying on-topic, or things getting a little out of hand, or patches dying as they get submitted and then ignored. I want to make sure that their concerns are met and we resolve those problems, which are really more human-scale problems than technical problems. If we can do that, then these people I think will come back. That will just make it easier for all of us to do our jobs. s: I can see how like the mailing-list interface with threads and things like that… if you think about it, there’s the mailing list, and then there’s the bug tracker, where you’ve got like “This is broken”, “Maybe there’s a patch needed for this”, but there isn’t really that trackable middle ground that says “Here’s an idea that I’ve been thinking about,” and maybe that’s why some of these ideas are getting dropped. Can you share a little bit about how you’re starting to keep track of the different balls that are in the air, the different things that you have on the go, to make sure that those things aren’t falling through the cracks? j: That’s an excellent question. I guess you might say that one of my tasks as maintainer is to keep an eye on these more inchoate or intangible things that people request or ask for that aren’t necessarily bugs, so they don’t necessarily belong in the bug database. They are ideas that get discussed on the mailing list, but as you say, when the thread gets cold, the idea might get lost. So what I do is: when these come up, I have an Org-mode file that I try to track all of the ideas or the things people have suggested or expressed concerns about to make sure that something gets done. Either they get addressed or they get knowledgeably closed–to say “No, leave it. We considered it but we’re not going to do it.” You’ll see that in my Org file here I have many top-level categories. There are things here, issues pertaining to the community; initiatives – things that I might want to begin but are not being done now; analysis is a little sub-project within Emacs to become more aware about potential bugs or performance problems that we might have, things that aren’t in the bug tracker. And under each of these, I have several keywords that are labeled as “Project”. I have a project here for making improvements within our community. One of the things I did, for example, was ask you to be our ambassador and keep us all aware of things outside of core development. Making sure responsibility for more aspects of Emacs development is distributed more evenly among the contributors so we don’t get burn-out. ELPA and changing the way we use ELPA… I’d like us to rely more on ELPA, have it be an easier and more facile way for people to get code into Emacs. And of course also the release schedule. Under these projects, then, I have various TODOs. Often–especially if they have a name here in parentheses–it means it links directly to a mailing-list article. If I hit a key to visit the link, it will pop me over to the emacs-devel mailing list and then show me that message in the context of its discussion. Or I could also then again press A T and then see it. (Although that didn’t work because I only have A T configured to work in the current group. Anyway.) But that gets me back to the original discussion about why I created that task. s: So I can see how this gives you an overview of the different initiatives that are going on, and lets you review this to follow up with the things that might be falling between the cracks. j: Right. The first line of activity I do is to follow the mailing list and anything that needs to be responded to there. I also like to read through the diffs log and see what changes have been made to Emacs, to see if there’s anything that needs more attention. I

You can listen to 2015-12-10 Emacs Chat: John Wiegley on maintaining Emacs and how you can help online on Radio and Podcast. Open the player on this page to stream the available audio.

2015-12-10 Emacs Chat: John Wiegley on maintaining Emacs and how you can help is an episode from Emacs Chat.

The episode duration depends on the source podcast feed and may not always be available.

This episode was published on Dec 10, 2015.

Yes. Use the heart button on the episode page to add it to your favorite episodes list.

Yes. This page shows related episodes from Emacs Chat when more episodes are available from the podcast feed.