Idea: advanced window transparency

[stjepan]

Screenshot: http://fs.tistory.com/attach/13/1097330232.png

Vista's "Add Gadget" window looks great. There's blurring so that you can see what's behind the window and see what's on the window. Reflections make the window pretty and realistic.

With Compiz (and Beryl) we can get blurring, reflections and Vista-like decorations. But what about transparency? Is it possible to make all windows look like "Add Gadget" window, entire transparent with a GTK2 engine?

I've heard that RYX has made a transparent GTK2 engine, but it seems lost...

[RYX]

The engine is not lost, but the whole thread on the old beryl-forums is unfortunately lost (it was the number three or four result on google when searching for "gtk engine" ... :().

The main problem of the engine is that it is very hackish and I think either needs some modifications in compiz or in gtk to work for all applications. There is a workaround for non-working apps, but it is too hackish for every day use ...

If you like to join working on the engine, your are happily welcome. The whole topic is very rarely documented but it's not really difficult. I can give you a brief overview of what I have "discovered" so far, if you like.

Check this screenshot (http://forum.go-compiz.org/viewtopic.php?t=404) for an example ...

(Edit: Looks like my server is currently down, we have a heavy storm going on in Berlin ... and it's a local hoster)

:)

[amgeex]

With Compiz we can make whole windows transparent, right? And we have a decorator plugin. So couldn't the decorator plugin be modified to apply transparency to the decoration?

[RYX]

The decorator only draws the window borders and we already have transparency-settings for gtk-window-decorator (see /apps/gwd/ in gconf-editor) ...

:)

[amgeex]

Oh well... Then lets see that gtk-engine code of yours! :lol:

[RAOF]

Doesn't GTK 2.10 have a "is compositing working" function? Because gnome-terminal has composited transparency when loading under compiz, and fake transparency when loading under metacity.

A hypothetical GTK engine could check that, and write to an ARGB buffer if possible or just an RGB buffer if not, right?

[FunkyM]

Doesn't GTK 2.10 have a "is compositing working" function? Because gnome-terminal has composited transparency when loading under compiz, and fake transparency when loading under metacity.

A hypothetical GTK engine could check that, and write to an ARGB buffer if possible or just an RGB buffer if not, right?

You got it. Transparency per application, but blur would be a global effect though and apply to all transparent windows.

Instead of hacking around something at the wrong place, people should always try to see where any idea/issue could be fixed upstream.

There are still parts in the GNOME desktop that have not yet made the transition to composite support (such as the gnome-panel; gnome-terminal is a counter-example) and that is where such features have to be implemented.

From what I read, David is already thinking about a clean way for regular applications to interact with compiz in order to provide useful stuff like controlling the blur for their application windows or fancy animations etc.

[RYX]

Oh well... Then lets see that gtk-engine code of yours!
I won't make the same fault twice ... that code is evil and it shouldn't be released (again). I can tell you how it works, but publishing the code will mean people show it around and other people think it is ready and working. Then they use it as if it was a final release and complain about Linux-desktop being unstable and so on ...

I've seen it last time - most people ignore the warnings and notes I put into the description and started using it as if it was ubuntulooks ... and that's bad for the "face" of the Linux-desktop.

I really agree with FunkyM and can confirm what he says: It needs work on the application-side to make the engine work. Especially apps like gnome-panel and firefox behave either strange or immediately crash on start (while gnome-terminal indeed works great with transparency). Apps need to be generally created with a depth of 32-bit to make them transparent - and that's simply not the case ...

(Though I don't think that hacking around in the engine is the wrong place if you want to have transparent widgets ... compiz doesn't draw the widgets, so upstream is the wrong place to begin imho.)

:)

[mikedee]

It needs work on the application-side to make the engine work. Especially apps like gnome-panel and firefox behave either strange or immediately crash on start (while gnome-terminal indeed works great with transparency). Apps need to be generally created with a depth of 32-bit to make them transparent - and that's simply not the case ...

Isn't it the case that most apps generally render a background or canvas and then start pasting widgets into it? If this is true then the background would have to be painted transparent which would probably cause all kinds of other problems (ie gaps in normal apps).

As far as I could see when messing around with mozilla and widgets, this was the basic problem. You could not render anything with a html tag because this tag automatically renders a body which is opaque (traditionally grey, but I think firefox uses white these days). Drawing directly onto a xulwindow was fine because that widget IS rendered transparently.

Is this roughly the problem as you see it?

[RYX]

Isn't it the case that most apps generally render a background or canvas and then start pasting widgets into it?
Most apps don't draw a single pixel, the engine does all drawing. As far as I can say, Gtk (i.e. the gtk theme-engine) generally draws a gray background pixmap and then renders the widget on it. My approach overdraws the gray background with transparent color before rendering the widget. If the window has an rgba-colormap, the background is "invisible" in those areas. If no rgba-colormap is available, the background is black.

The better approach would be (imho) to make the background-pixmap transparent, I think that CAN be done by the engine (obviously in the gtk-engine's set_background_pixmap-function), but I was unable to do it (due to lack of knowledge about gtk/gdk).

As far as I could see when messing around with mozilla and widgets, this was the basic problem. You could not render anything with a html tag because this tag automatically renders a body which is opaque (traditionally grey, but I think firefox uses white these days). Drawing directly onto a xulwindow was fine because that widget IS rendered transparently.
The biggest problem I had with firefox is that everything that should be black is fully transparent (within the mozilla-widget's contents). That means fonts, outlines, ... everything is transparent (while the page-background is colored as it should be). To work around this, the engine could explicitly check for a mozilla-widget and handle it different from default widgets.

I think setting the default colormap for all widgets to rgba (that's what my approach does) is the source of the crashes when several apps get started (firefox, twf, ..). Making the background really transparent would be the better way and fit better into the gtk-architecture ...

:)

[RYX]

I really would like to know the gtk-developer's opinion on this. So if there is a "nice" way to make the gray/white bg of a GdkWindow (because that's what "behind" the widdget) transparent.

Is anyone in contact with some gtk-devs?

:)

[delfick]

so then anymore on this ???

(i really want this :D, i even have the first engine RYX made :D)

[stjepan]

DELFICK IS HERE!
Welcome :)

Delfick, can you post that engine?

Is anyone in contact with some gtk-devs?

I bet MacSlow can help!

[delfick]

DELFICK IS HERE!

hehehe :D.....thnx for the warm welcome :D

Welcome :)

i take it you haven't been reading this thread http://forum.go-compiz.org/viewtopic.php?p=3253#3253

Delfick, can you post that engine?

sry, RYX says it shouldn't be spread to other people........(for good reasons, it was buggy, and made some programs a little dodgy)

[FunkyM]

I think this changeset applies to this thread:
http://gitweb.freedesktop.org/?p=xorg/app/compiz.git;a=commit;h=969e9c10c5445505796a53245407 eb6e135d64bf

Allows applications to control blur through WM hints. (compiz always > beryl on those things)

Most (maybe all?) of the GTK+ stuff uses Cairo, thus it is possible to allow transparency wherever you like. However once again, this has to be implemented on the application side (vs. compiz side).

[RYX]

I bet MacSlow can help!
I think I know MacSlow's opinion on this topic back from the old beryl-thread. He didn't seem to be the biggest fan of the engine ... (I bet it was too ugly and hackish for his taste :D ...)

Most (maybe all?) of the GTK+ stuff uses Cairo, thus it is possible to allow transparency wherever you like. However once again, this has to be implemented on the application side (vs. compiz side).
Sorry - once again I have to partly disagree ... :)
The gtk-engines do all drawing for 99% of all applications ... so the engine is the point to start if you want transparency for common widgtets (button, toolbar, menu, scrollbar ...). You're right - most gtk-engines use cairo and can be drawn with transparency. Things like the mozilla-widget or terminal-emulators are an exception because they are owner-drawn (i.e. app-paintable) ... these would require some changes in those apps. The gnome-panel can be easily themed to allow transparency with the modifications I described ...

The partial blur is another thing - the latest changes allow compiz to only blur (or add other effects to) a certain area of a window instead of applying all effects to all transparent areas.

[MacSlow]

It is correct that for general transparency the gtk-theme-engine would be the right spot to work on. Of course this makes only sense for composited environments. Thanks to the new calls regarding compositing in gdk starting with 2.10.x and the right window-manager in place a theme-engine could "branch" its drawing code. One for composited environments with transparent-parts where wanted/needed, and one for the non-composited case. Still this is not as easy as it may sound. I had a go at this once (patched around a recent version of the murrine-engine) and ran into some surprising drawing-glitches that were anything but obvious to resolve.

But... there is always a but :)... this (general widget transparency) is just a super tiny thing compared to a really powerful "next generation UI-toolkit". If you know just a bit about the stuff available in OS X (Quartz Compositor) or Vista (WPF or .NET 3.0 or some other buzzword) just doing general widget-transparency is a joke, really.

I'd say... either we do the full thing or don't do it at all. I want a toolkit that allows me to alpha-blend video-surfaces, GL-surfaces, cairo-surface, pango-text, widgets... have them stack/layer over each other... twirl/twist/fade/transform them around... animated of course if I want... and on top of that I want this to be editable in a pipeline or treelike (XML-ish?) via a tool like inkscape or so... maybe with some glade-like UI-resource XML-file at the end I just can load-in at runtime and hookup my event-callbacks and such... maybe even have certain parameters of the UI still runtime-controlable.

Remember my "begun the bling-wars have" blog-entry form months ago.

This is so much serious and hard work, it will not be doable by just a bunch of bling-horny people like us on a spare-time schedule. If we would have this though, instantly all newly written apps could benefit from it... but then it is hardly likely to be anywhere near API-compatible to current gtk+ or so. Trying to be API-compatible to gtk+ would make it even harder and probably cause more headaches than necessary.

On Linux you need a developer to make something like cairo-clock run. On OS X or Vista you 'just' need a designer (thanks to the available frameworks and front-ends using these frameworks). Here's a humble try to come up with a good comparison: It's like the step from assembly-programming to programming in high-level languages. You can achieve the same results in both, but you'll be much more productive and faster in the high-level language.

We just don't have anything on the free desktop that even comes close to the frameworks _and_ front-ends on the proprietary systems.

Gee... what a bloody bitching post this one became :(

Best regards...

MacSlow

[RYX]

I agree with MacSlow (though it is a little disappointing message, indeed :)). I have no real ambition to create a transparency-engine myself and want to concentrate on the Screenlets ...

If anyone starts working on a new UI-toolkit with "next-generation-desktop" in mind, I would love to help ;) ... but something like that would need very good coordination, planning and developers - and most likely some company support.

:)

[delfick]

*blown away*......

that sounds cool :D

.....

anyone have a magic lamp so we can wish this possible ?? :D :P......

[imnotpc]

anyone have a magic lamp so we can wish this possible ??

I've been rubbing my magic lamp but I keep getting Barbara Eden. Darn...

[ioannis]

I want a toolkit that allows me to alpha-blend video-surfaces, GL-surfaces, cairo-surface, pango-text, widgets... have them stack/layer over each other... twirl/twist/fade/transform them around... animated of course if I want... and on top of that I want this to be editable in a pipeline or treelike (XML-ish?) via a tool like inkscape or so... maybe with some glade-like UI-resource XML-file at the end I just can load-in at runtime and hookup my event-callbacks and such... maybe even have certain parameters of the UI still runtime-controlable.

why not coordinate this with one (or both) of the popular toolkit developer groups? Isn't KDE "promising" big steps towards the "next generation desktop experience". And I've heard many saying that GTK needs to reinvent itself.

It needs to start somewhere/somehow, otherwise it will never happen.

[PaK]

why not coordinate this with one (or both) of the popular toolkit developer groups? Isn't KDE "promising" big steps towards the "next generation desktop experience". And I've heard many saying that GTK needs to reinvent itself.

It needs to start somewhere/somehow, otherwise it will never happen.

Developers Developers Developers(tm) ;-) Right.. it shuld start from somewhere but there is alredy propably some projects similar to that what macslow said. But they are on begining of their way to get all necessary stuff. Look at compiz.. initial cvs import was on 8 feb 2006 almost one year ago ande where it is now ? And only one developer, but what developer :) All just i want to say is starting from somewhere without support of GTK/CAIRO/OpenGL real developers will bring project to dead end IMHO. Good design and implementation is hardly depend on real pro developers.

On GTK mailing list there was post that GTK have some problems with developers... there is to few specialists.

KDE team is focused on doing KDE 4, they are basing KDE 4 on qt 4.2(3), that is much step forward from QT 3, but its still need probably much work to get the POINT. Probably ingeeners from TT are working on design of QT 5 that will have that features. So making QT based framework to do that stuf imho isnt good idea, because TT will bring that features sooner or later.

[ioannis]

... but then it is hardly likely to be anywhere near API-compatible to current gtk+ or so. Trying to be API-compatible to gtk+ would make it even harder and probably cause more headaches than necessary.MacSlow

another good point here. Sorry for nitpicking MacSlow's post :) It's only because he captured a wide range of problems one might face in the pursue of the holy grail of toolkits.

API-compatibility is something that shouldn't be much of a concern. One approach is to amend the current API with new sets of functions, most of which might even duplicate functionality, but in a slightly different way. I think that's what Microsoft has been doing throughout their OS evolutionary steps. IMHO it's a recipe for disaster. The preferred way would be to build an entirely new API (using your experience from the previous one) with all the bling / bells & whistles you can think of (at the time) and provide compatibility libraries, i.e. re-implement the legacy API on top of the new one. Yes, the legacy apps may not look as great as the new ones, but it makes everyone happy and gives them time to switch gradually. And it shouldn't result in any noticeable performance issues, since it would mostly be a simple wrapping involved and plus the new API would most possibly make better use of the hardware (via accelerated backends?). If I'm not mistaken this is Apple's approach as well.

And I'll agree with the Pak in that one man can make a difference :-D, but since not everyone is David Reveman (or <put here your favourite developer>), coordination with the developers of existing toolkits is probably the best way to go.

I don't know about you, but I feel excited and am optimistic about the future of linux desktop :)

[RYX]

I really like the direction this discusion is heading into now :)

On the one hand I agree with MacSlow - good things need time and sweat, but on the other hand I am a big fan of the idea of a "real" 3d desktop ... I just don't know a thing about GL-coding. My favorite UI-toolkit should be able to use any rendering backnd, make use of hw-acceleration, offer full access to modern compositing features and allow full integration/interaction with the (to be written) graphics/effects/filter-layers of the graphics-server ...

I wrote several small widget-toolkits (the first one about 15-20 years ago with TurboPascal 6 under DOS :)) in various languages so it is not an entirely new topic for me but the design of a revolutionary toolkit would require more than some hobbyists (here I again fully agree with MacSlow) ...

(*sigh*)

:)

[FunkyM]

We got (GNOME-wise):

- GTK+ with GDK drawing using Cairo
- Cairo with Glitz backend using HW acceleration
- Compiz for "enabling" compositing and offer other stuff to client apps
- GStreamer for Multimedia support

Now, ain't it a matter of extending GDK/Cairo/Compiz/GStreamer to allow for more rich user interfaces?

Afaik., things like Apple's CoreImage/CoreVideo/Quartz coding stuff is widely not possible atm. as there is no such simple API for it in the above mentioned libraries yet.

Maybe it's easier to talk using concrete examples.

Imagine an iTunes-like Rhythmbox 3D "flyby" cover art browser.

It would require an simple API to draw images to an offscreen image using perhaps different blendmodes and transparency, all this preferably with HW acceleration by Cairo/Glitz.

Is a complete new toolkit really required for this in the short/mid-term? (Of course we need a new toolkit in the long-term, maybe GNOME 3.0...)

[RAOF]

...

Imagine an iTunes-like Rhythmbox 3D "flyby" cover art browser.

It would require an simple API to draw images to an offscreen image using perhaps different blendmodes and transparency, all this preferably with HW acceleration by Cairo/Glitz.

Is a complete new toolkit really required for this in the short/mid-term? (Of course we need a new toolkit in the long-term, maybe GNOME 3.0...)

You should be able to do this in Cairo at the moment, as far as I can tell (I've never tried). I'm not sure how well accelerated it is, though.

MacSlow, couldn't you just layer an API on top of Cairo to do your XML-foo? Most of those effects sound like they're goals of the Cario API

[Matias]

Just passing by.
Have you took a look to this guy: http://njpatel.blogspot.com/index.html

Their mookups looks great. Although they are just mookups it sound like he has some code and cool ideas. Can be helpful.

[ioannis]

Going back to the original post, the screenshot shows a semi-transparent window with (possibly gaussian) blur effect.

Many are suggesting cairo-based gtk-theme engines. I've been trying to find out if cairo includes gaussian blur filters. It seems that it doesn't. There is a gaussian filter enumeration, though I don't know what that's used for (anti-aliasing maybe?)

Does anyone know if cairo can do gaussian blurred fill/stroke?
If yes, how?

thanks

PS: something tells me that the only way to do this (with usable performance) is on OpenGL 2.0 compliant GFX cards, using GLSL.

[RAOF]

...
PS: something tells me that the only way to do this (with usable performance) is on OpenGL 2.0 compliant GFX cards, using GLSL.

Pixel shaders rock :). Although you can apparently do less cool blur with multi-texturing in ways which don't destroy performance.

[delfick]

anyone seen this

http://njpatel.blogspot.com/2007/02/while-i-have-your-attention_07.html

:D

Appearance
As you can see from the previous mock ups, there are some funky themes in there! That leads me to my next point, which is this: I love having a composited desktop, Awn doing its thing, Beryl throwing windows all around the screen, cairo-clok ticking away, however, in my day-to-day tasks, I am left a bit disenchanted, therefore, as a side project for me to show my cairo/gtk prowess(:p), I am also going to start work on a gtk theme that takes advantage of the composited environment. Now, please bear in mind that a theme has to be used all around your desktop, and I am guessing that any theme that uses extensive cairo drawing + transparency is going to have an effect on your cpu, but I still think its worth it as a proof-of-concept.

:D

[RYX]

I gotta say - a little late for a proof-of-concept, but looks nice ... ;) ... (And not more usage of CPU - the GPU gets more work instead :) ...). Transparent buttons and UI seem to be one of the most wanted eyecandy-related things on the composited desktop. Maybe I should pick up my work again (question to myself)?

P.s: Afaik cairo doesn't support any blurring (yet?). That's why MacSlow suggested creating an effects-library back in the days when this discussion was made in old compiz-quinn forums ...

:)

[ioannis]

RYX, yep I think you should (if you have the time of course). I could help. I've been trying to get involved in one of these eye-candy projects but found it time consuming to bring my self up to speed with their current state. Starting something new might be a better approach.

Is there any library that provides abstract filter effects such as blur. Cairo has filtering operations and they might add blur later one, but I think it's not what we are looking for. The current filters change the way the things cairo draws look. Things like anti-aliased fonts, or patterns. The effect we are looking for is a distortion-like one, which, if I'm not mistaken, is not covered by the cairo API. It could be added as a transparency/opaque property (at least as seen from the user's point of view). I'm not very familiar with the terminology, but I think it involves rendering layers and how one layer modifies the visual representation of the one below it. I've seen some talks about adding pipelines in cairo, but don't know if it's related.

It would be nice if this is added to something like Cairo, to maximise applicability. Many projects will benefit from this and let's face it, it's better to focus on developing one presentation library rather than branching out into multiple tiny projects, each one working on their own little niche domain.

[flargen]

If you want to blur transparent areas of windows you can make use of the Compiz blur plugin by setting the correct window hint. But I assume you mean blurring parts of widgets on the window, not behind...

[RYX]

I think in a few months compiz will evolve to much more than just a window-manager. David's plans lead to the ultimate compositing and window-manager where things get taken off the x-server and get placed into the compositor (at least where it makes sense) ... Maybe it is soon possible to use compiz for much more than just drawing and maybe it will include filters/effects for everything ...

I still share MacSlow's ideas of a general effects-library - maybe this will be a nice side-product of the upcoming changes to compiz. It would surely make sense to place effects like blur/* into a separate library ...

(Concerning the gtk-engine I think I can't resist to play with it and try some new approaches from time to time - but I have more important things to do first :) ... it is not working well, anyway)

:)

[ioannis]

That is true. David has great plans for the future of compiz. "the future is bright" (tm) :-P

I guess you can use the blur plugin to achieve the general look and feel the original screen-shot has on this topic (from Vista). So that's good :)

[mikedee]

It would surely make sense to place effects like blur/* into a separate library ...

In a way it already is, but you can only control where the plugin blurs under your transparent window, you cannot blur on top of it, although I suppose that would be easy.

[darkmatter]

There is an OpenGL toolkit in active development (along with gtk bindings), that may just end up being the answer to many 'prayers' as it were :)

http://clutter-project.org/

Clutter is an open source software library for creating fast, visually rich graphical user interfaces. The most obvious example of potential usage is in media center type applications. We hope however it can be used for a lot more.

Clutter uses OpenGL (and soon optionally OpenGL ES) for rendering but with an API which hides the underlying GL complexity from the developer. The Clutter API is intended to be easy to use, efficient and flexible.

Though young, Clutter currently features the following:

* Object oriented design via GObject with a familiar GTK+ like API.
* Scene-graph of interface elements with multiple layers, grouping, transparency, scaling and rotation.
* Frame based animation engine providing path interpolation, transitions and other custom effects via Behaviours and Timelines.
* Custom Pango renderer providing efficient internationalized text rendering.
* Support for media playback with GStreamer, Cairo graphics rendering and GTK+ embedding are available via add-on libraries.
* Language bindings for Perl and Python.


Looks promising.

[RYX]

I think that's what MacSlow mentioned somewhen. Looks really promising :D ... The demos look very smooth with close to no cpu-usage. But it doesn't appear like a replacement for gtk.

(To compile the python-bindings on Ubuntu the "python-gtk2-dev"-package is needed)

:)

[MacSlow]

Greetings everybody!

I was at FOSDEM this weekend (even got persuaded to give a talk *uff*) and was lucky to meet core developers of cairo, X.org, drivers and gtk+ among many other great people.

Without going into too much details I can tell you that things like cairo and GL on X will get significantly faster (less stressing on the CPU and busses), gtk+ will become "bling-aware".

BTW, it is correct that libclutter is not meant to replace gtk+ or extend it. It is targetted to be a handy library for one-app-on-one-screen kiosk-like programs based on GL, cairo, gtk+/gobject. Furthermore libclutter does not provide means to do 3D/OpenGL in a simplified way. They use GL only to make their 2D-UI stuff fast.

Of a bit broader scale might be pigment (the "UI-engine" behind Elisa from Fluendo). pigment, like libclutter, also offers python-bindings and has concepts of 3D in its design. I might investigate the possibility to base a future cairo-clock on it to make my life easier.

Best regards...

MacSlow

[RYX]

Thanks MacSlow for keeping us updated! You saved my day ... :D:D:D (bling-aware-gtk sounds like a dream)

(After playing a little more with libclutter I realized that it's design is very similar to FlashPlayer9's Display-architecture. It could be great for creating simple and lightweight "one-app-desktops" for children or older people - combined with a touchscreen it could become quite handy.)

[*no_smiley_can_express_that_much_happiness*]