State of Neovim 2022

the

state

of neovium the

state

of

neovim

there is the corporate headquarters of

neovim

um last year I gave a talk about how neovim should operate uh in 2019 in Japan I gave a talk about why neovim exists and why it made the decisions it made and liked it what and what those options should be in the future, and now this talk looks at the progress since last year, which was when I outlined some ideas to harness the energy and popularity of the project that has happened. in the last year and I think it's actually probably an effective anecdote as a sort of hint at how much has changed in the last almost nine years and the current momentum of the project is a true story.

I was in a It was kind of a kind of big meeting with Amazon leaders in Seattle a few weeks ago and at the end of the meeting, my managers, managers, managers, they saw a sticker on my laptop and they were like, "Oh, Neovim, you use that, I love it and I said yes, I use it um and what I learned from this interaction was number one I probably shouldn't rule out so quickly that I got a tattoo on my face with the neovim logo to make it more obvious that. that. I'm closely involved with project number two.

It's great that Amazon leadership is technical enough to use projects like many of them and number three is much more popular and doing well than I expected and is finding. its paths into nooks and crannies that I didn't expect and we're probably going to peak, I would say in the next year or two in popularity because I don't know how we can keep doubling users every year, it's just a modal text editor, so, yeah, that, that, that, that, that, that, that's, that's a clue to the current status of nine of them. I'm on Keys uh online on Twitter and GitHub.

My name is Justin MK. I have been with Neovium Car Maintainer since almost the beginning. I still remember, even before the news of him starting to talk to Thiago on Reddit and, like, talking about potential strategies to get this thing off the ground. To this day, the type of readmeat text you see in the neovim repository is almost unchanged from those conversations we had on Reddit in late 2013. um and it's surprising that things have changed. I really had to change a lot from those initial conversations, despite the fact that we are actually continually, like reexamining things on a regular basis, my favorite model, my favorite, to think about how the project works is simulated and kneeling, and a part key to simulated annealing is that there are cooling phases, so although there are really chaotic phases like uh evolutionary, there's also kind of a cooling stage, um and uh, you know, that's where, like the signal is extracted from the noise of chaos and um, so, so, this is reexamining things, it's important for the project and for us to, in a way, continue to take shape as a coherent direction for the project instead of just having, as you know, a big bag of features, so the topics of this talk are purpose, progress and plan, so again we're going to briefly reexamine the question of why, uh, which one do you know?

It was examined much more deeply than in 2019, but we always want to revisit it at least once a year. Having a conference for the text editor is a good place, it's a good mechanism to induce that kind of re-examination and then we'll talk about tangible things like what we actually accomplished, last year's ideas and plans, how they worked, how they manifested. and then briefly, hopefully, we'll have enough time to talk. the plan, but as you know, it's hard for me to adapt to these things, so the purpose, the purpose, like the landing page, has a kind of more specific list of goals from similar projects, but it's really the elevator.

The pitch for the project is this extensible and usable Mantra Vim that you see stuck in various places on the website, then Forks focused on extensibility and usability on GitHub and hyperextensible event base technologies that actually exchange, but The coincidences are these. three types of uh Vim usable extensible core ideas all three have the same weight um and the reason why it's okay that I want to insist on this is because it seems that sometimes this is forgotten, especially in like discussions about project direction that they happen in PRS and stuff like that and that image at the bottom that you see is like a science fiction like representation of some kind of vehicle or something harvesting the useful minerals from an asteroid um thank you and um uh yeah so uh working them is basically like mining asteroids I really like this analogy um like we bring the raw material like there's four ways you can get useful things from an asteroid um there's four approaches and we're using all of those approaches in any one of them number two bringing back raw material is kind of analogous to forking and we did it, sending patches to Upstream is like doing manufacturing on the asteroid itself or somewhere in space and um and processing stuff on site and then getting products out of it which is like the API and transporting the application the asteroid physically back to Earth um that's pretty much uh that's very uh it's obvious The parallels with the incorporation of these are the core activities of the neovim project, but then we return to the purpose , removing the focus on the Vim part of the Mantra, since one of the main purposes is actually to preserve.

I like the useful parts of them, so when people just start, I've seen some more like Cavalier talk about uh, you know, we don't need this old stuff like here and there, like demscript sucks or they like it. Regular expressions, do you know why we need them? Well, I mean, this is one of the three pillars of the project and, if we take it away and just forget about it, then we have to really like it. reexamine really do what's what again is the goal what's the direction uh I think I explained why we should preserve the parts of Vim, the useful parts of it in 2019, so I won't go into that. here but but but but but neovim's audience is narrower than them and vs codes themselves that seem to try to target the entire universe, they in particular target the entire universe in a clumsy way, um, you have easy Vim, which actually it's like it's kind of an oxymoron, but it's trying to target people who don't want a modal text editor, which doesn't really make any sense, and then it also mentions again, referring to this slide from 2019, just the same thing.

The amount of useful content beneath the ugly surface of legacy is actually captured in something called the Lindy effect which is on Wikipedia and is a description of the idea that they like things that are really old. They've been around even longer than things that just started to exist, and the reason for this tends to be that they actually have a lot of value, so I didn't know about this term in 2019, but I did my best to sort of map it out and describe it. and it's cool that it actually has a name so in the neovan project we're targeting the second half of this somewhat famous quote as if the tool is something like mold mold us uh we choose to do this if we're going to convert the tool into another thing then uh, we like the first question is, well, what's the point? um and so, this is an intentional restriction, it's intentional that Well, we're not like in the main project focusing on creating an integrated experience because that tends to have this kind of treadmill testing.

Treadmill testing is tasks like keeping up with every random language server configuration, that's a treadmill, that's a lot of work. It's a big distraction and takes away all the endless work we have to do to create a text editor in this way. I think of it as a towel from The Hitchhiker's Guide to the Galaxy, where it's just like that, it's really simple and minimalist, but super versatile, a multi-tool that you can carry with you in all kinds of contexts and, in fact, it's like when I listen to people talk about having a fully integrated and ready-to-use experience.

This is very confusing to me anyway because today you can combine neovim with vs code. This works fine. I use it all the time at work. I simply install this plugin that is linked here and this was one of the original purposes any of them existed for. I mean, the reason I got excited about the project in late 2013 was because I wanted to embed it in Atom, but instead of Atom we now have vs Code and whatever the next wave is, we'll continue to surf those waves, that's right. the power of having a hosted model that is flexible and can work in many different contexts, so that it answers directly, just this, to have the modesty to reexamine this question directly, as if from scratch.

We addressed the original milestones, the labor controlled terminal of 2016. but then do we need to continue existing? What is the point? Or um, I still see a lot of similar topics that interest me personally and that's what keeps me coming back to work on the project, things that interest me. everyone working on the project has things they are personally interested in, docked date UI is something that isn't really addressed anywhere else. I think there's still a lot of interesting things that will come out of that, including Neo Neo of those are the two neovim own that you'll use, the decoupled UI and that's sort of analogous to a compiler being a self-hosted compiler, it's going to be an interface of self-hosted user and then, uh, just the huge Cambrian explosion of little plugins that we've seen in the last year or two.

I'm seeing concrete, real results from that, like for example the get signals loop plugin that's written entirely in place of Neo of them by Lewis, um, that's it. best in class gets a sign complement for them or neozim period so this is an example of how this type of simulated annealing process is generating real tangible useful results um and then personally you know I like to have a I breathe from them. script like me, I actually like it, I like the flexibility of them and emacs, but I want a language that doesn't work against me, isn't obsessed with syntax, new forms of syntax, I just want a language that is small, minimal and actually like it has really consistent mechanisms, um, so Lua is like an alternative, uh, and it kind of fills that gap for me and in terms of core development, any of them, the Neovium development model . continues to render as really useful features of the editor uh instead of just you know, uh, obsessed with syntax, uh, language development, um, where it actually produces real features, status column, uh, zero command height, things So, it seems that people like to prefer working on this code base. instead of the code base of them, this was all part of the original purpose of the oven and continues to be relevant today and finally, uh, the nanny story is still totally the same as the early days and it looks really promising and I.

I'm excited to see what comes of that, but also you know again, like a self-examination, what about all these other super amazing text editors? Well, I think they all converge on the same idea. Hitchhikers arrived in the galaxy. towel, the minimal multi-tool, um, they're all converging on this idea, so I don't really care at all which one comes first or, optimally, like that's it, they're all part of the simulated annealing process, where, like um There are different types of chemical reactions happening and then we're converging on the North Star CS code and the eclipse and Jeff's brain fleet are going in that direction from a different angle, but it's not exactly the same thing and and in fact I would propose that , like um, like a unique way of gauging whether something is an IDE or a text editor because some people, like I think, struggle with trying to figure out, like they know, where to draw the line.

I think it might be interesting to think of the line being drawn as if you could download the source code of the project and compile it from source in five to ten minutes, then you could probably call it a text editor if you can't, but I can't do that. if it takes hours or a server farm then it's probably an idea, so the next stage of the talk is to talk about concrete, concrete progress in the last year. This is always a very fun slide. GitHub downloads, which are just people clicking on the release page or maybe Bots also clicking on the release page.

I don't know how GitHub calculates those downloads, but now it's 2.3 million. A year ago I was at 170k, so it has more than doubled. there's a clear sort of exponential curve um uh there Homebrew isn't quite as exponential but it's still healthy growth um 476 uh thousand downloads or installs per year versus 300,000 per year last year an Overflow survey voted us the publisher of most liked text or IDE two years in a row andsomething interesting also if you dig into the raw numbers of the stack overflow poll you can actually see that the number of votes in that section for anyoneof them increased year over year and the absolute number of votes for Vim actually decreased by about 10 10 10 percent about 10 15 year over year, so I don't know what's going on there, but it's interesting to see all these signs like they're continuing hinting that this project is relevant and on Reddit. 44,000 members, this is interesting because in 2014, when there weren't any of them, the Vim subreddit only had 30,000 members total and now it's tripled to uh or or quintupled to uh uh 150,000 members, so this flies in the Front of all the people who were complaining and wouldn't stop and you'll still find them on Hacker News, um saying that neovim divides the community and will divide the resources well, it actually turns out that this is not a zero sum calculation. because if you significantly multiply the amount of enthusiasm around modal editing and them and Vim, sort of code base and usage, you multiply the activity and mind sharing around that and you get a bunch more developers in general, working on both projects you actually won a lot and it turns out that's what happened and I think Nelson probably deserves a fair amount of credit for that and the other interesting thing that I like is a password for like and and I also confirm something that I intuited years ago on the Vim Dev mailing list, but something that, uh, Brown admonished me for it, but something I said was and I still have this position, which is like users who aren't really actively using any of the features of their products that were made in the last 10 or 20 years don't count, they're useless, they don't care if they like it, they accidentally opened it to edit a git commit message, they don't care if it's nano or Vim or whatever.

They're not real users, they're not, they're not actually moving the needle on anything at all, they're not increasing development activity, they're not increasing Mind share or buzz, they're not filing bug reports. nothing, they're useless, um and you can, this is like a perfect example of it: the Vim subreddit has three times as many members, but there are only like 178 of them actually active, while there are actually more active users on the subreddit neovim and it only has a third of the total membership size of uh so this is another sign also that a lot of vem enthusiasts from the Vim editing model uh are using neovim and I'm experiencing this even like on lazy channels at work you're probably coming across this where it's like but where do all these new Avengers come from?

Does anyone actually use them? It's surprising to see that it's kind of shocking, but uh and and then again, uh in terms. From a modest self-examination, let's talk about the risks, the risks I see are, you know, too much activity that is not channeled in a youthful way, something that Fred Brooks called conceptual integrity, you need that kind of unique, approximately unique human consciousness. as an architect it's like the narrow waist or the conduit through which concepts flow, um, if we can't control that and we just get a huge bin of Walmart deals with features that don't work well. together, then we'll become less relevant and and and and we'll suffocate under our own weight and another risk is that, like Vim patches, they're not really horizontally distributed yet. not at all we rely a lot on one person to do it which has been awesome uh but it's definitely a risk and then our Windows support we still need like some Windows developers like to take the time and care to make sure That things work fine on Windows, although we have received help from 3n.

I want to call it and the final risk is that as major plugins, if they decide not to support any of them, then it's a big risk, but that risk. it is actually decreasing. I never, ever expected this two years ago. I did not do it. This wasn't even a remote possibility for me. I thought plugins like uh we would depend on vimscript for a long time for many important plugins. but this risk is reducing, but I think Vent Fugitive is a good example of a plugin that would be what I would really miss if it wasn't compatible with any of them, so um um but and uh regular when it's like that, when you know, something like putting the flag in the ground, uh, for the vem9 script, uh, that's when we actually had it, something that we had liked to move towards, but didn't like to officially declare, was that actually.

We're just going to go completely towards Lua, this made the decision really easy and has produced this extreme popularity that we've been talking about so far. I think also the popularity you know it is. It's definitely largely due to YouTube personalities like TJ and Primogen, and the built-in LSP, the just-works model of Lua and Vim plugins, which shouldn't be taken for granted, as it required a lot of careful work, most of it. part. by Bjorn and then Lua himself deserves credit for the explosion here. Louis is the 4th fastest growing language on GitHub, it's 10x more popular than vimscript and I think it deserves credit.

It has that. concept economy has that structural integrity um that I referred to before uh the concepts are composable like closures work the same everywhere the functions you can pass them everywhere uh of course, the tables are ubiquitous and then the Co routines that have the has covered like a lot of these concepts that, like in recent articles by these PL people, they're looking at these things like coroutines and modules, um. Maybe I didn't understand the point of this blog post, but it seems that way to me. compares favorably to Lou alua has mature answers to some of these some of these stories that are mentioned as the next big thing for programming languages ​​to adopt such concrete features that here is a list of concrete features that we deliver in

2022

in particular one which I think is exemplary for the neovim extensibility story and that part of our Mantra is the UI attach, vim.ui attaches the Lua interface to something to the UI protocol that the RPC plugins have had access to.

For years this is now also open just like we exposed libuv, just like we exposed Vim to plugins, now UI events are exposed to in-process Lua plugins and it didn't take long for Falky to take advantage of this. in this very interesting plugin called neistat which finalizes them where it simply replaces each UI component exposed by the UI protocol um file type.lua was a really deep performance job thanks to Christian and others for working on that Diagnostic by g panders spell nanny from Lewis and uh and uh and uh clickable status line um um and then the semantic highlighting for LSP just landed on master and then Teresa from MDOC um text letter as a platform .

I talked about this in 2019, but the OS failed, so it's up to the browser to be the GUI OS and it's up to the text editors to be the OS on your terminal. Work we've done in

2022

for that's the vim.ui namespace. We continue to develop it so that um plugins. you can override the core functionality of the text editor UI and vim.secure is a way to support local workspace configuration and potentially, maybe you know we could introduce the project concept around that somewhere moment in the future, the funding for neogum is uh, zero overhead, it is used only to fund the developers who work full time for at least a month on the project.

We've funded over 10 developers like this since 2014 and the 2022 feature list that was delivered from the funding is the vem line script to pull the transpiler, that's going to be what we're looking at as a way to continue mining the Upstream work of them, particularly the text test coverage and the runtime plugins, so it's important work and then, um, the thread, uh, has been doing some really deep and careful work on the performance, low level performance, RPC mappings uh and um also working on an advanced form of FAMU virtual text um fully done made the incremental command preview uh open that up so it's extensible to plugins, so plugins now they can provide uh preview of previewable commands uh, as you type any command and clickable status line Global status line, so and uh prop progress in Tracer itself, we don't run Tree Sitter, but it's a major dependency of ours and it's also something that we look at carefully because we kind of, uh, have bet the farm on it in a sense um now it's part of emacs um GitHub and sourcecraft and they're using it um other companies like Amazon use it um and uh I think Vim will end up using it too despite their best efforts to prevent it, and now also, some Upstream contributors are showing great interest in updating the innovations of Christian and others for any of the tree sitters and then using them for the bleeding. and folding concepts, so how well did we do channeling the energy since last year when we speculated on a strategy to do this we are no longer a small group so it is more difficult to correct mistakes, but at least as a group large, if we can channel the energy, then that can be good and it can be a big advantage of being a large group if you are able to channel the energy in a useful way and you do it through documentation, so we have included a whole lot of documentation with Lua documentation, love documentation, uh, nanotees, Lua guide, but now we also have a parser for vemdoc, this is a big problem because it gives us a huge amount of time and we no longer have to answer the question. oh, should we move to Markdown for documentation?

We don't really have to answer that question right now because instead of some old crappy helper script we now have Lua plus trees that understand the structure of the document and give us like a useful AST and that means we can generate really nice looking HTML and we can give that AST information to neovim itself if so, here's a before and after review of our HTML documents and now, because that shitty Ox script is gone, I got contributions from other people. Christian added the ability to have annotated code blocks and now we have syntax highlighting built into the help docs and this works even in the terminal, even in the editor, if you enable the trees that are in the help docs.

Progress on automation.goc is the Highlander of automation work and the old but very important neovim refactoring work. We now have fully automated code formatting applied by CI and it can also be used locally so you never have to think about formatting new event contributors no longer think about formatting it's like working in a code base go um and uh dendergarch I also contributed to a really cool improvement to Vim's patch script that makes the patches it generates much more useful because it aligns the project format from them to the neovim format, so now we get much more useful diffs generated.

We also now have several people who can cut pitches, so this is a problem solved. we never no longer have that single point of failure um and uh uh continuing the movement the simulated kneeling analogy the uh moving fast this was part of our strategy that I described in last year's talk the general idea is like the Gnomes from South Park collect Underpants, which is like chaos. You do something with Thomas Matandas to make something happen and then you get something good at the end, so actually chaos is step two I think, but are we moving? too quickly, I'm seeing posts now on reddit about people feeling overwhelmed because of the pace of change and, uh, this is kind of an open question, actually, it's like an open backwards compatibility question, um, but , you know, yeah.

I don't really have much to say there um uh another part of our strategy last year was like we had a model for deprecating things in an orderly way and so we presented deprecated things for that and we're also writing a formal guide for Lua, a compatibility Supported API that will be analogous to, but not exactly the same as, the RPC API contract, so this is like a rough idea of ​​the progress we made since last year's talk, like uh, I still haven't managed to fully implement remote Tui and there are some Open questions about where is the shape of the standard library, the Lewis Deadlib API, so going forward, the strategy and plan.

I hope to combine the currents to channel the energy into a narrow waste simply by writing more things with clearer guidelines. I think I am seeing very clearly that there is a lot of confusion within us. our contributions and our PR discussions about what kind of names should be used for APIs, when we should use new modules, so the challenge here is to have clear guidelines to answer these questions and hurt cats, this is this. Iskind of like what I feel when I see a lot of the concepts that we have in our system, like we control the neovim system, we don't control everything outside of any of them, but we control the system within the neovim universe, so there's really no an excuse for us to have multiple similar but slightly different concepts, um, like it's not necessary to be like that, we just need to write the guidelines and another part of our plan is like uh, I think now that we have like uh, we have a way forward , a clear path for what the documentation is like.

We'll see and like what our documentation practices are. um, it's all like the help system is like the source of truth. We're probably going to move, get rid of the wiki, um, we're not going to have things like this. we know random pages on the website and we're not going to have random readme files everywhere so we've established that and then the next step I think is to build the Lewis of I think that's what the next 12 months really that's on what I'm really going to be thinking about and working on um and overall yeah, the idea is to scale, just continue to scale like the documentation strategy is like a key part of this, how do you scale? a group of humans that are like AIS that you don't really control, it has to be through clear, clear documentation and guidelines and kind of an ownership structure and we've started doing that, like PRS auto-tagging. and we have some ownership of the code of the defined files and then we also continue, we spend time thinking about the economics of the concepts and, because in advocating for that, I've found it difficult to convince people. that this matters, but I think it has a direct effect on discoverability and usability, which is one of the three core themes of our combination with the goal of our project, um, I think that's mainly, yeah, the other big concrete idea is the remote control.

I think Tui is interesting because it is analogous to a self-hosted compiler.