https://blogs.gnome.org/tbernard/2023/07/26/rethinking-window-management/ Skip to content Space and Meaning Tobias Bernard's GNOME Blog Menu and widgets Search for: [ ] [Search] Recent Posts * Rethinking Window Management * Berlin Mobile Hackfest * Post Collapse Computing Part 4: The Road Ahead * Post Collapse Computing Part 3: Building Resilience * Post Collapse Computing Part 2: What if we Fail? Archives * July 2023 (1) * June 2023 (1) * January 2023 (1) * October 2022 (2) * August 2022 (1) * July 2022 (1) * May 2022 (1) * September 2021 (2) * August 2021 (2) * July 2021 (1) * June 2021 (3) * April 2021 (1) * March 2021 (1) * March 2020 (1) * January 2020 (1) * December 2019 (4) * September 2019 (1) * April 2019 (1) * March 2019 (1) * February 2019 (1) * December 2018 (1) * October 2018 (2) * August 2018 (1) * May 2018 (2) * April 2018 (2) * January 2018 (1) * November 2017 (1) Pages * About [2023-04-26-17-29-38-132-825x510] Rethinking Window Management Window management is one of those areas I'm fascinated with because even after 50 years, nobody's fully cracked it yet. Ever since the dawn of time we've relied on the window metaphor as the primary way of multitasking on the desktop. In this metaphor, each app can spawn one or more rectangular windows, which are stacked by most recently used, and moved or resized manually. [floating-windows2-1-1024x576]Overlapping windows can get messy quickly The traditional windowing system works well as long as you only have a handful of small windows, but issues emerge as soon the number and size of the windows grows. As new windows are opened, existing ones are obscured, sometimes completely hiding them from view. Or, when you open a maximized window, suddenly every other window is hidden. Over the decades, different OSes have added different tools and workflows to deal with these issues, including workspaces, taskbars, and switchers. However, the basic primitives have not changed since the 70s and, as a result, the issues have never gone away. While most of us are used to this system and its quirks, that doesn't mean it's without problems. This is especially apparent when you do user research with people who are new to computing, including children and older people. Manually placing and sizing windows can be fiddly work, and requires close attention and precise motor control. It's also what we jokingly refer to as shit work: it is work that the user has to do, which is generated by the system itself, and has no other purpose. Most of the time you don't care about exact window sizes and positions and just want to see the windows that you need for your current task. Often that's just a single, maximized window. Sometimes it's two or three windows next to each other. It's incredibly rare that you need a dozen different overlapping windows. Yet this is what you end up with by default today, when you simply use the computer, opening apps as you need them. Messy is the default, and it's up to you to clean it up. What about tiling? Traditional tiling window managers solve the hidden window problem by preventing windows from overlapping. While this works well in some cases, it falls short as a general replacement for stacked, floating windows. The first reason for this is that tiling window managers size windows according to the amount of available screen space, yet most apps are designed to be used at a certain size and aspect ratio. For example, chat apps are inherently narrow and end up having large amounts of empty space at large sizes. Similarly, reading a PDF in a tiny window is not fun. [forge2-1024x576]GNOME 44 with the "Forge" tiling extension. Just because windows can be tall and narrow doesn't mean they should be :) Another issue with tiling window manager is that they place new windows in seemingly arbitrary positions. This is a consequence of them not having knowledge about the content of a window or the context in which it is being used, and leads to having to manually move or resize windows after the fact, which is exactly the kind of fiddling we want to avoid in the first place. https://blogs.gnome.org/tbernard/files/2023/07/ ipad-tiling-shortened.webm More constrained tiling window managers such as on iPadOS are interesting in that they're more purposeful (you always intentionally create the tiling groups). However, this approach only allows tiling two windows side-by-side, and does not scale well to larger screens. History This topic has been of interest to the design team for a very long time. I remember discussing it with Jakub at my first GUADEC in 2017, and there have been countless discussions, ideas, and concepts since. Some particular milestones in our thinking were the concept work leading up to GNOME 40 in 2019 and 2020, and the design sessions at the Berlin Mini GUADEC in 2022 and the Brno hackfest in 2023. [2023-04-26-11-24-43-741-1024x768]Tiling BoF in Brno during the HDR hackfest. Left to right: Robert Mader, Marco Trevisan, Georges Stavracase, Jakub Steiner and Allan Day (remote), Florian Mullner, Jonas Dressler I personally have a bit of a tradition working on this problem for at least a few weeks per year. For example, during the first lockdown in 2020 I spent quite a bit of time trying to envision a tiling-first version of GNOME Shell. [prometheus-1024x640]2020 mockup for a tiling-first GNOME Shell. More mockups in the OS mockups repo on Gitlab. Problems with our current tiling GNOME has had basic tiling functionality since early in the GNOME 3 series. While this is nice to have, it has obvious limitations: * It's completely manual * Only 2 windows are supported, and the current implementation is not extensible to more complex layouts * Tiled windows are not grouped in the window stack, so both windows are not raised simultaneously and other windows get in the way * Workspaces are manual, and not integrated into the workflow [tiling-status-quo-1024x576]Because tiled windows are are currently mixed with overlapping floating windows they're not really helping make things less messy in practice. We've wanted more powerful tiling for years, but there has not been much progress due to the huge amount of work involved on the technical side and the lack of a clear design direction we were happy with. We now finally feel like the design is at a stage where we can take concrete next steps towards making it happen, which is very exciting! Get out of my way The key point we keep coming back to with this work is that, if we do add a new kind of window management to GNOME, it needs to be good enough to be the default. We don't want to add yet another manual opt-in tool that doesn't solve the problems the majority of people face. To do this we landed on a number of high level ideas: * Automatically do what people probably want, allow adjusting if needed * Make use of workspaces as a fully integrated part of the workflow * Richer metadata from apps to allow for better integration Our current concept imagines windows having three potential layout states: * Mosaic, a new window management mode which combines the best parts of tiling and floating * Edge Tiling, i.e. windows splitting the screen edge-to-edge * Floating, the classic stacked windows model https://blogs.gnome.org/tbernard/files/2023/07/mosaic-open-close.webm Mosaic is the default behavior. You open a window, it opens centered on the screen at a size that makes the most sense for the app. For a web browser that might be maximized, for a weather app maybe only 700x500 pixels. https://blogs.gnome.org/tbernard/files/2023/07/mosaic-maximize.webm As you open more windows, the existing windows move aside to make room for the new ones. If a new window doesn't fit (e.g. because it wants to be maximized) it moves to its own workspace. If the window layout comes close to filling the screen, the windows are automatically tiled. https://blogs.gnome.org/tbernard/files/2023/07/mosaic-tile2.webm You can also manually tile windows. If there's enough space, other windows are left in a mosaic layout. However, if there's not enough space for this mosaic layout, you're prompted to pick another window to tile alongside. https://blogs.gnome.org/tbernard/files/2023/07/mosaic-tile3.webm You're not limited to tiling just two windows side by side. Any tile (or the remaining space) can be split by dragging another window over it, and freely resized as the window minimum sizes allow. https://blogs.gnome.org/tbernard/files/2023/07/ mosaic-vertical-tile.webm There are always going to be cases that require placing a window in a specific position on the screen. The new system allows windows to be used with the classic floating behavior, on a layer above the mosaic/ tiling windows. However, we think that this floating behaviour is going to be a relatively uncommon, similar to the existing "always on top" behavior that we have today. There's of course much more to this, but hopefully this gives an idea of what we have in mind in terms of behavior. New window metadata As mentioned above, to avoid the pitfalls of traditional tiling window managers we need more information from windows about their content. Windows can already set a fixed size and they have an implicit minimum size, but to build a great tiling experience we need more. [huge-clocks-1024x576]Some apps should probably never be maximized/ tiled on a 4K monitor... One important missing piece is having information on the maximum desired size of a window. This is the size beyond which the window content stops looking good. Not having this information is one of the reasons that traditional tiling window managers have issues, especially on larger screens. This maximum size would not be a hard limit and manual resizing would still be possible. Instead, the system would use the maximum size as one factor when it calculates an optimal window layout. For example, when tiling to the side of the screen, a window would only grow as wide as its maximum width rather than filling exactly half of the screen. In addition, it'd be helpful to know the range of ideal sizes where an app works best. While an app may technically work at mobile sizes that's probably not the best way to use that app if you have a large display. To stay with our chat example, you probably want to avoid folding the sidebar if it can be avoided, so the range of ideal sizes would be between the point where it becomes single pane and its maximum usable size. Ideally these properties could be set dynamically depending on the window content. For example, a spreadsheet with a lot of columns but few rows could have a wider ideal size than one with lots of rows. Depending on apps using new system APIs can be challenging and slow -- it's not easy to move the entire ecosystem! However, we think there's a good chance of success in this case, due to the simplicity and universal usefulness of the API. Next steps At the Brno hackfest in April we had an initial discussion with GNOME Shell developers about many of the technical details. There is tentative agreement that we want to move in the direction outlined in this post, but there's still a lot of work ahead. On the design side, the biggest uncertainty is the mosaic behavior -- it's a novel approach to window management without much prior art. That's exciting, but also makes it a bit risky to jump head-first into implementation. We'd like to do user research to validate some of our assumptions on different aspects of this, but it's the kind of project that's very difficult to test outside of an actual prototype that's usable day to day. If you'd like to get involved with this initiative, one great way to help out would be to work on an extension that implements (parts of) the mosaic behavior for testing and refining the interactions. If you're interested in this, please reach out :) There's no timeline or roadmap at this stage, but it's definitely 46+ material and likely to take multiple cycles. There are individual parts of this that could be worked on independently ahead of the more contingent pieces, for example tiling groups or new window metadata. Help in any of these areas would be appreciated. This post is summarizing collaborative work over the past years by the entire design team (Allan Day, Jakub Steiner, Sam Hewitt, et al). In particular, thanks to Jakub for the awesome animations bringing the behaviors to life! Posted on July 26, 2023Author Tobias BernardCategories Development Tags Design, GUADEC, Shell, Tiling 11 thoughts on "Rethinking Window Management" 1. [46cee8] Anselm Schuler says: July 26, 2023 at 15:00 This is a very cool idea! Reply 2. [c6a7a3] lele says: July 26, 2023 at 16:18 Seems like a novel and interesting idea! Although I am not sure about these 2 ideas: > You can also manually tile windows. If there's enough space, other windows are left in a mosaic layout. **However, if there's not enough space for this mosaic layout, you're prompted to pick another window to tile alongside.** > If a new window doesn't fit (e.g. because it wants to be maximized) it moves to its own workspace. Those behaviors seem kinda unpredictable/inconsistent since they depend on the current layout situation. Although I guess that can only be judged when actually trying it out in practice... Any way, the state of the union of the design was certainly a cool talk a, good job! Hopefully, some stuff makes it into gnome sooner(tm) than later :) Reply 3. [f956d4] Alex says: July 26, 2023 at 17:09 Have you looked at https://github.com/paperwm/PaperWM ? You are proposing different things, but I think PaperWM is still worth looking at- it works very well, and it's very simple to use... Reply 4. [6368c0] Alex says: July 26, 2023 at 17:19 I like to recommend you having a look at XMonad and it's configuration options. Reply 5. [e57ae5] NaheemSays says: July 26, 2023 at 19:12 I have an ultrawide monitor that is flat. This sets the unique problem that not all the screen is equally visible from normal working distance. While mosaic is interesting, what I find myself doing is moving the window I am working on to the centre of the screen before working on it if it is for anything other than a very brief interaction. Reply 6. [362b6a] Kye says: July 26, 2023 at 19:58 This blog post made me remember of a desktop concept I found a few years ago: https://desktopneo.com/ maybe you can get some inspiration from this Reply 7. [c06e3f] Cameron says: July 26, 2023 at 20:16 I have always wanted to organization of a tiling window manager, but I haven't wanted to learn all these keyboard shortcuts and relearn how I use my desktop. That is why I think this Mosiac solution is genius, and I'm hoping to see it in a future GNOME release, or as an extension earlier on. Reply 8. [7998e3] Nick Richards says: July 26, 2023 at 20:34 The opportunistic creation and destruction of workspaces for full screen windows was something we had in Moblin/MeeGo Netbook 'back in the day' and worked very well in practice and user testing (if any of that really old stuff is helpful). Looking forward to seeing a much more advanced version of this in future, this all looks very promising. One behaviour I've been using a lot recently in GNOME is 'popping out' a picture in picture video window and using the 'always on top' action on the window to have that work 'properly'. It's a bit ungainly currently and would be nice if there were OS safeguarded ways of having that work more pleasingly, even in the floating case. Reply 9. [722a7a] Nico says: July 26, 2023 at 20:36 Interesting and well explained! "If a new window doesn't fit (e.g. because it wants to be maximized) it moves to its own workspace." This could break my workflow (insert XKCD here). Shortcuts for window navigation and workspace navigation are different, which is why I generally avoid workspaces unless I have to repeatedly switch between groups of windows. Having window actions lead to workflow creation could impact current usage and might bring confusion. It might be a topic worth exploring in user testing. Reply 10. [e56c78] Francis Whittle says: July 26, 2023 at 20:46 How does this play with multi monitor? Specifically the idea of automatically generating a new workspace for some cases - would not want all my monitors switching to another desktop when one needs a maximised window, or to have such behaviour constrained to the primary monitor only. Reply 11. [cec927] Matt Park says: July 26, 2023 at 20:52 I've been thinking about this a lot for the past few years as I want to build something similar to a news reader, but can also function as a dashboard for office big screen. Maximizing space is key. Well I ran into a game on hackernews called "Prosperous Universe" -- I won't link it so I don't get put in the penalty box by antispam. The game UI could be subtly tweaked to function as a full OS UI. You get a tiled UI with floating windows "temporary buffers" which you can use sort of ephermally, or you can drag them into your tile setup for more permanance. You then have multiple named desktops you can fire up, but all the ephemeral floating windows follow you desktop to desktop. Nearly perfect. Try it out, go through the tutorial... actually I think they host them on youtube so you could technically take a peak without signing up for the game. If you do try it out, let me know / email me, certainly it will provide some nice food for thought, and I'd be curious to know what you think. Reply Leave a Reply Cancel reply Your email address will not be published. Required fields are marked * [ ] [ ] [ ] [ ] [ ] [ ] [ ] Comment * [ ] Name * [ ] Email * [ ] Website [ ] [Post Comment] Wordpress Hashcash needs javascript to work, but your browser has javascript disabled. Your comment will be queued in Akismet! [ ] [ ] [ ] [ ] [ ] [ ] [ ] D[ ] This site uses Akismet to reduce spam. Learn how your comment data is processed. Post navigation Previous Previous post: Berlin Mobile Hackfest Proudly powered by WordPress