mutter/doc/cookbook/effects.xml

<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
 "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">

<chapter id="effects" xmlns:xi="http://www.w3.org/2003/XInclude">
  <title>Effects</title>

  <epigraph>
    <attribution>Roger Zelazny, from <citetitle>Prince of Chaos</citetitle>
    </attribution>
    <para>Don't wake me for the end of the world unless it has very
    good special effects</para>
  </epigraph>

  <section id="effects-introduction">
    <title>Introduction</title>

    <para>Effects modify an actor's appearance, such
    as how it is positioned, colored and textured.</para>

    <para>The Clutter API for effects contains several
    abstract classes you can subclass to create your own effects.
    It also contains several built-in effects you can use to
    modify the visual appearance of actors in a variety of ways.</para>

    <para>The recipes in this section of the cookbook cover how to create
    your own effects as well as how to apply Clutter's effects.</para>

    <section>
      <title>Creating effects using the abstract effect classes</title>

      <tip>
        <para>One of the original design goals of Clutter was to abstract
        the complexity of GL. However, the effects API partially circumvents
        these abstractions, to give you finer-grained access to the
        graphics pipeline. Therefore, if you want to write your own effects,
        some understanding of Cogl, OpenGL, and general graphics programming
        is essential.</para>
      </tip>

      <para>Each abstract effect class is tailored to modifying different
      aspects of an actor, as explained below:</para>

      <itemizedlist>

        <listitem>
          <formalpara>
            <title><type>ClutterEffect</type></title>
            <para>If you're just using the Clutter and Cogl APIs to
            decorate an actor, this is simplest type of effect to
            implement.</para>
          </formalpara>

          <para>Subclassing <type>ClutterEffect</type> enables you to
          "wrap" how an actor is painted, by injecting some code before
          and/or after the actor's own <function>paint()</function>
          implementation.</para>

          <note>
            <para>This is the preferred way to modify how an actor is
            painted, short of creating your own actor subclass.</para>
          </note>

          <para><emphasis>Subclasses of
          <type>ClutterEffect</type></emphasis>:</para>

          <itemizedlist>

            <listitem>
              <formalpara>
                <title><type>ClutterOffscreenEffect</type></title>

                <para>Use this class as a basis if you need GL textures
                for your effect.</para>
              </formalpara>

              <para>GL textures are required for effects which need
              an offscreen framebuffer. The offscreen framebuffer is
              used to store a modified rendering of an actor (e.g.
              with its colors altered or with deformed geometry).
              This buffer is then redirected to a texture in the
              stage window.</para>

              <para>An example is <type>ClutterBlurEffect</type>,
              which uses a GLSL fragment shader to blur an
              actor's appearance in an offscreen framebuffer.</para>

              <para><emphasis>Subclasses of
              <type>ClutterOffscreenEffect</type></emphasis>:</para>

              <itemizedlist>

                <listitem>

                  <formalpara>
                    <title><type>ClutterDeformEffect</type></title>

                    <para>Use this base class if you want to modify
                    an actor's geometry, at the level of individual
                    vertices.</para>
                  </formalpara>

                  <para><type>ClutterDeformEffect</type> removes the
                  complexity of dealing with vertex-based deformations
                  at the OpenGL level, instead enabling you to easily plug
                  a deformation callback into the graphics pipeline.</para>

                  <para>If you are writing your own deform effects,
                  a good example to work from is
                  <type>ClutterPageTurnEffect</type>.</para>

                  <para>There is also a
                  <link linkend="effects-custom-deform">recipe which
                  explains how to implement a simple custom deform
                  effect</link> (a page fold).</para>

                </listitem>

                <listitem>
                  <formalpara>
                    <title><type>ClutterShaderEffect</type></title>

                    <para>Use this if you want to apply custom
                    GLSL vertex or fragment shaders to your actors.</para>
                  </formalpara>

                  <para>Writing <type>ClutterShaderEffects</type> gives
                  you very fine-grained control over the GL pipeline.
                  However, this makes them the most complex
                  effects to implement.</para>

                  <tip>
                    <para>If you want to write your own GLSL shaders, the
                    <ulink url="http://www.opengl.org/documentation/glsl/">GLSL
                    specification</ulink> is a good starting point.</para>
                  </tip>

                </listitem>

              </itemizedlist>

            </listitem>

          </itemizedlist>

        </listitem>

      </itemizedlist>

    </section>

    <section id="effects-introduction-using-the-built-in-effects">
      <title>Using the built-in effects</title>

      <para>Clutter comes with a number of built-in effects
      which can easily be applied to your actors. This section
      explains how to do this.</para>

      <para>First, create an actor. For this
      example, we use a texture loaded with an image:</para>

      <informalexample>
        <programlisting>
/* filename could be set from command line or constant */
gchar *filename;

/* create a texture */
ClutterActor *texture = clutter_texture_new ();

/* ...set texture size, keep aspect ratio etc... */

/* NB ignoring missing file errors here for brevity */
clutter_texture_set_from_file (CLUTTER_TEXTURE (texture),
                               filename,
                               NULL);

/* ...add texture to the stage... */
        </programlisting>
      </informalexample>

      <para>Next, create an instance of an effect; here, we're
      creating a <type>ClutterColorizeEffect</type> with a pink tint:</para>

      <informalexample>
        <programlisting>
ClutterColor *pink = clutter_color_new (230, 187, 210, 255);
ClutterEffect *effect = clutter_colorize_effect_new (pink);
        </programlisting>
      </informalexample>

      <para>Finally, apply the effect to the actor:</para>

      <informalexample>
        <programlisting>
clutter_actor_add_effect (texture, effect);
        </programlisting>
      </informalexample>

      <para>The result in this case is an image colorized with
      a pink tint, like this:</para>

      <screenshot>
        <mediaobject>
          <imageobject>
            <imagedata format="PNG"
                       fileref="images/effects-built-in.png" />
          </imageobject>
          <alt>
            <para>Applying a <type>ClutterColorizeEffect</type>
            to a texture loaded with an image (drawing by
            Madeleine Smith)</para>
          </alt>
        </mediaobject>
      </screenshot>

      <para>The same set of steps applies for any of the built-in
      Clutter effects. Your own custom effects classes should also
      behave in a similar way: constructors should return
      <type>ClutterEffect</type> instances so your effect can
      be added to an actor through the standard API.</para>

      <para>One further thing worth mentioning is that because an
      effect is a GObject, any properties you expose for your effect
      can be animated  via implicit animations,
      <type>ClutterAnimator</type> or <type>ClutterState</type>. For
      example, the <type>ClutterPageTurnEffect</type> can be animated
      by manipulating its <varname>period</varname> property. An example
      of how to do this for your own effect is given in the
      <link linkend="effects-custom-deform">custom deform effect
      recipe</link>.</para>

      <para>The full code for the <type>ClutterColorizeEffect</type>
      example is below.</para>

      <example id="effects-introduction-example-1">
        <title>Applying a <type>ClutterColorizeEffect</type> to
        a texture loaded with an image</title>
        <programlisting>
<xi:include href="examples/effects-built-in.c" parse="text">
  <xi:fallback>a code sample should be here... but isn't</xi:fallback>
</xi:include>
        </programlisting>
      </example>

    </section>

  </section>

  <section id="effects-custom-deform">
    <title>Creating and animating a custom <type>ClutterDeformEffect</type></title>

    <section>
      <title>Problem</title>

      <para>You want to deform an actor's geometry: for example,
      to make it appear stretched, twisted or folded.</para>

      <para>This recipe demonstrates how to do this with a simple page
      fold effect, which folds one half of the actor over its other half.</para>
    </section>

    <section id="effects-custom-deform-solution">
      <title>Solution</title>

      <para>Subclass <type>ClutterDeformEffect</type> and
      implement a <function>deform_vertex()</function> function
      to modify the actor's vertices.</para>

      <para>The signature for <function>deform_vertex()</function>
      is:</para>

      <informalexample>
        <programlisting>
void
deform_vertex (ClutterDeformEffect *effect,
               gfloat               width,
               gfloat               height,
               CoglTextureVertex   *vertex);
        </programlisting>
      </informalexample>

      <para>The <varname>width</varname> and <varname>height</varname>
      are the width and height of the target material, stored in
      the offscreen buffer. Usually the target material's size will
      match the actor's transformed size; however, if the effect
      implements <function>create_texture()</function>, the target
      material's size may differ from the actor's transformed size.</para>

      <para>The <varname>vertex</varname> contains the position
      and color of a vertex, to be deformed by your effect.
      Your <function>deform_vertex()</function>
      function should modify the member variables of this
      <type>CoglTextureVertex</type> in place. Usually, this will
      mean modifying the <varname>x</varname>, <varname>y</varname>
      and <varname>y</varname> member variables of the vertex,
      which describe its position in 3D space.</para>

      <para>The example function below, taken from
      <link linkend="effects-custom-deform-example-2">the
      full example</link>, applies a transformation to vertices falling
      in the "right-hand" half of the actor (i.e. vertices with an
      <varname>x</varname> value greater than or equal to half the
      width of the actor).</para>

      <informalexample>
        <programlisting>
static void
cb_page_fold_effect_deform_vertex (ClutterDeformEffect *effect,
                                   gfloat               width,
                                   gfloat               height,
                                   CoglTextureVertex   *vertex)
{
  CbPageFoldEffectPrivate *priv = CB_PAGE_FOLD_EFFECT (effect)->priv;

  /* the rotation angle is modified by the percentage progress of the fold,
   * as represented by the period variable
   */
  gfloat radians = (priv->angle * priv->period) / (180.0f / G_PI);

  /* rotate from the center of the actor on the y axis */
  gfloat adjusted_x = vertex->x - (width / 2);

  /* only rotate vertices to the right of the middle of the actor */
  if (adjusted_x >= 0.0)
    {
      vertex->x = (vertex->z * sin (radians))
                  + (adjusted_x * cos (radians))
                  + width / 2;

      /* NB add 1 to z to prevent "z fighting"; otherwise, when fully-folded
       * the image has "stripes" where vertices from the folded part
       * of the actor interfere with vertices from the unfolded part
       */
      vertex->z = (vertex->z * cos (radians))
                  + (adjusted_x * sin (radians))
                  + 1;
    }

  /* adjust depth of all vertices so they fit inside the actor while folding;
   * this has the effect of making the image smaller within the texture,
   * but does produce a cleaner fold animation
   */
  vertex->z -= width / 2;
}
        </programlisting>
      </informalexample>

      <para>Note that this effect has two properties set in its
      constructor or through setters:</para>

      <orderedlist>
        <listitem>
          <para><varname>angle</varname>, representing the angle of
          the full fold; for the actor to fully fold in half, this
          would be set to 180.0</para>
        </listitem>

        <listitem>
          <para><varname>period</varname>, representing the percentage
          of the fold to apply</para>
        </listitem>
      </orderedlist>

      <para>As well as rotating the vertex, the
      <function>deform_vertex()</function> function also shifts
      the <varname>z</varname> coordinate "up" by 1
      (towards the viewpoint) for vertices on the right-hand side of the
      actor. This is so that the "folded over" vertices
      are above vertices on the left-hand side. Without this small
      shift, the vertices interfere with each other, which can cause striping
      artefacts.</para>

      <para><emphasis>All</emphasis> vertices are also shifted "down",
      so that the the folding part of the actor remains within the texture.
      Otherwise the part which is folding may be clipped to the allocation of
      the actor.</para>

      <para>This effect can now be applied to an actor, using the
      approach
      <link linkend="effects-introduction-using-the-built-in-effects">outlined
      in the introduction</link>. The result looks like this when
      <varname>period</varname> is set to 0.25 and <varname>angle</varname>
      to 180.0 (i.e. the page is folded by 45 degrees):</para>

      <screenshot>
        <mediaobject>
          <imageobject>
            <imagedata format="PNG"
                       fileref="images/effects-custom-deform.png" />
          </imageobject>
          <alt>
            <para>Applying a custom <type>ClutterDeformEffect</type>
            to a texture loaded with an image</para>
          </alt>
        </mediaobject>
      </screenshot>

      <para>Because the effect is a GObject which exposes its
      properties, it can easily be animated, as described in
      <link linkend="effects-custom-deform-discussion-animating">the
      discussion section</link>.</para>

    </section>

    <section id="effects-custom-deform-discussion">
      <title>Discussion</title>

      <para>A deform effect processes an actor as follows:</para>

      <itemizedlist>

        <listitem>
          <para>The actor is divided into a series of
          triangular tiles. The number of
          horizontal and vertical tiles is configurable;
          more tiles implies more vertices. See
          <link linkend="effects-custom-deform-discussion-tiles">this
          section</link> for more details about tiles.</para>
        </listitem>

        <listitem>
          <para>The position of each vertex of each
          tile is then modified (or not) by the
          <function>deform_vertex()</function> function. In this
          function, you can change the vertex's position
          (<varname>x</varname>, <varname>y</varname>,
          <varname>z</varname> coordinates). You can also
          modify the color at the vertex if desired.</para>

          <para>The resulting deformed vertices are stored
          in an offscreen buffer.</para>
        </listitem>

        <listitem>
          <para>Once the deformation has been applied to
          all vertices, the content of the offscreen buffer
          is painted at the onscreen position of the actor.</para>
        </listitem>

      </itemizedlist>

      <para>You may find it useful to visualise this process by imagining
      your actor's surface as a net, composed of triangles. (Something
      like a fishing net, not a mathematical one.) At each corner of
      each triangle is a marble; and between each pair of corners
      is an infinitely flexible length of elastic. Moving a marble
      doesn't change the position of its neighbours; it just stretches
      or relaxes the elastic.</para>

      <para>In this analogy, the marbles are the vertices; and the
      surfaces between the marbles, bordered by triangles of
      elastic, are the tiles. More triangles (tiles) means more
      marbles (vertices).</para>

      <para>When you create a <type>ClutterDeformEffect</type>,
      think of it as specifying movements of marbles in the net.
      Changing the position of a vertex corresponds to moving a marble
      up/down (-/+ <varname>y</varname> position), left/right
      (-/+ <varname>x</varname> position) or away/towards
      you (-/+ <varname>z</varname> position) (ignoring color for the
      moment).</para>

      <para>Now imagine that you are asked to fold the whole net of
      marbles; but you can't just grab the edge of the net and pull
      it over: you can only move one marble at a time. However, once moved,
      each marble magically stays where you put it in 3D space.</para>

      <para>To do this, you could project where each marble would be if
      you could fold the whole sheet in one go; then move the
      marbles one by one to their projected positions. Even though
      you'd be moving the marbles one at a time, it would eventually
      look as though you'd folded the whole net with a single movement.</para>

      <para>When you write a <type>ClutterDeformEffect</type>, you have
      to accomplish a similar feat: change the shape of an actor
      by individually modifying the positions of points on its surface. In
      most cases, your <function>deform_vertex()</function> implementation
      can take advantage of an existing geometric transformation
      method to achieve this. (For example, the page fold in this recipe
      is based on equations from p.412 of <citetitle pubwork="book">Computer
      Graphics (C Version), 2nd Edition</citetitle> by Hearn and
      Baker, 1996.)</para>

      <section>
        <title>Customising the back material</title>

        <para>When you set up a deform effect, you
        can optionally specify a material to use for the "back" of
        any actor it is applied to.</para>

        <para>If you think of an actor as a sheet of paper with a
        picture on it, specifying a back is similar to turning the
        sheet of paper over (rotating it around the
        <varname>y</varname> axis) and drawing another picture on
        the other side. If you then folded or twisted the paper,
        you would be able to see parts of the pictures on both the
        front and back of the paper.</para>

        <para>Similarly, during deformation of an actor, if any
        vertices of the actor are deformed such that the actor's surface
        is folded or twisted over itself, parts of its back
        become visible. If you set a back material, you will see parts
        of that where the surface is folded over. If you don't set a back
        material, you will instead see mirror images of parts of the actor's
        front: as if the actor was flexible stained glass, rather than paper.
        You can see this if you watch the animation in
        <link linkend="effects-custom-deform-discussion-animating">this
        section</link>.</para>

        <para>The back material should be an instance of
        <type>CoglMaterial</type>. You can either create this via
        the Cogl API directly; or indirectly through the Clutter API
        (for example, by getting the material from a
        <type>ClutterTexture</type>). The code below gives an example
        of how to do the latter:</para>

        <informalexample>
          <programlisting>
<![CDATA[
/* create a texture */
ClutterActor *back = clutter_texture_new ();
clutter_texture_set_keep_aspect_ratio (CLUTTER_TEXTURE (back), TRUE);
clutter_actor_set_width (back, 400);

/* load image into texture (ignoring errors for brevity) */
clutter_texture_set_from_file (CLUTTER_TEXTURE (back),
                               back_image_file,
                               NULL);

/* get a handle to the texture's Cogl material */
CoglHandle material = clutter_texture_get_cogl_material (CLUTTER_TEXTURE (back));

/* cast the effect to ClutterDeformEffect and set its back material
 * to the handle
 */
clutter_deform_effect_set_back_material (CLUTTER_DEFORM_EFFECT (effect),
                                         material);
]]>
          </programlisting>
        </informalexample>

        <para>See the <type>ClutterDeformEffect</type> API reference
        for more details about back materials.</para>

        <para>Here's a screenshot of the
        <link linkend="effects-custom-deform-example-3">example</link>
        with the addition of a back material, folded at an angle
        of 60 degrees:</para>

        <screenshot>
        <mediaobject>
          <imageobject>
            <imagedata format="PNG"
                       fileref="images/effects-custom-deform-back-material.png" />
          </imageobject>
          <alt>
            <para>Applying a custom <type>ClutterDeformEffect</type>
            to a texture loaded with an image</para>
          </alt>
        </mediaobject>
      </screenshot>

      </section>

      <section id="effects-custom-deform-discussion-animating">
        <title>Animating a custom deform effect</title>

        <para>Clutter's animation API can animate any GObject which
        exposes its properties. In the case of the page fold effect,
        we can expose the <varname>period</varname> property using
        standard GObject property installation:</para>

        <informalexample>
          <programlisting>
/* GObject class init */
static void
cb_page_fold_effect_class_init (CbPageFoldEffectClass *klass)
{
  GParamSpec *pspec;
  GObjectClass *gobject_class = G_OBJECT_CLASS (klass);

  /* ...other class setup code... */

  /* expose the period as a GObject property */
  pspec = g_param_spec_double ("period",
                               "Period",
                               "The period of the page fold",
                               0.0, 1.0,
                               0.0,
                               G_PARAM_READWRITE);
  obj_props[PROP_PERIOD] = pspec;
  g_object_class_install_property (gobject_class, PROP_PERIOD, pspec);

  /* ...install other properties... */
}
          </programlisting>
        </informalexample>

        <para>We also add a <function>get_property()</function>
        implementation, as well as a setter (see
        <link linkend="effects-custom-deform-example-2">the full
        GObject implementation</link> for details).</para>

        <para>Then set up an animation for the property; in this case,
        using a <type>ClutterState</type>:</para>

        <informalexample>
          <programlisting>
ClutterEffect *effect = cb_page_fold_effect_new (180.0, 0.0);

ClutterState *transitions = clutter_state_new ();
clutter_state_set_duration (transitions, NULL, NULL, 500);

clutter_state_set (transitions, NULL, "unfolded",
                   effect, "period", CLUTTER_LINEAR, 0.0,
                   NULL);

clutter_state_set (transitions, NULL, "folded",
                   effect, "period", CLUTTER_LINEAR, 1.0,
                   NULL);
          </programlisting>
        </informalexample>

        <para>To start the animation, warp the <type>ClutterState</type>
        into its <emphasis>"unfolded"</emphasis> state, then set it to
        <emphasis>"folded"</emphasis>:</para>

        <informalexample>
          <programlisting>
/* this changes state instantaneously */
clutter_state_warp_to_state (transitions, "unfolded");

/* this starts an animation to the state */
clutter_state_set_state (transitions, "folded");
          </programlisting>
        </informalexample>

        <para>Note that the
        <link linkend="effects-custom-deform-example-3">full code
        sample</link> is slightly more complex, as it triggers state
        changes when a mouse button is pressed on the texture. There is
        also a third "partially folded" state (used to create
        the screenshot for the
        <link linkend="effects-custom-deform-solution">previous
        section</link>).</para>

        <para>Here's what the resulting animation looks like:</para>

        <inlinemediaobject>
          <videoobject>
            <videodata fileref="videos/effects-custom-deform.ogv"/>
          </videoobject>
          <alt>
            <para>Video showing animation of a custom deform effect
            on a texture</para>
          </alt>
        </inlinemediaobject>

      </section>

      <section id="effects-custom-deform-discussion-tiles">
        <title>Tiles</title>

        <para>A <type>ClutterDeformEffect</type> divides the actor
        being deformed into a number of tiles: the larger the number
        of tiles, the larger the number of vertices to be manipulated
        by the effect. Increasing the number of tiles increases the number of
        vertex computations required, which can slow down animations;
        at the same time, finer-grained tiles can make an effect appear
        smoother, particularly when animated.</para>

        <para>Most of the time, the default number
        of tiles in the <varname>x</varname> and <varname>y</varname>
        axes should suffice. You can get the current number of
        tiles associated with an effect with:</para>

        <informalexample>
          <programlisting>
<![CDATA[
guint x_tiles;
guint y_tiles;

/* effect must be a subclass of ClutterDeformEffect */
clutter_deform_effect_get_n_tiles (CLUTTER_DEFORM_EFFECT (effect),
                                   &x_tiles,
                                   &y_tiles);
]]>
          </programlisting>
        </informalexample>

        <para>However, if an effect produces jerky or fragmented output,
        you want to tweak the number of tiles. Use the
        <function>clutter_deform_effect_set_n_tiles()</function> function
        to do this:</para>

        <informalexample>
          <programlisting>
/* 64 tiles in both axes */
guint x_tiles = 64;
guint y_tiles = 64;

clutter_deform_effect_set_n_tiles (CLUTTER_DEFORM_EFFECT (effect),
                                   x_tiles,
                                   y_tiles);
          </programlisting>
        </informalexample>

      </section>

    </section>

    <section>
      <title>Full example</title>

      <para>This example consists of three files:</para>

      <itemizedlist>
        <listitem>
          <para><link linkend="effects-custom-deform-example-1">A header
          file</link> for the <type>CbPageFoldEffect</type> GObject.</para>
        </listitem>
        <listitem>
          <para><link linkend="effects-custom-deform-example-2">The
          code file</link> implementing <type>CbPageFoldEffect</type>.</para>
        </listitem>
        <listitem>
          <para><link linkend="effects-custom-deform-example-3">A short
          sample application</link> which applies a <type>CbPageFoldEffect</type>
          instance to an actor and animates the fold when the actor is
          clicked.</para>
        </listitem>
      </itemizedlist>

      <para>As Clutter effect subclasses are written using GObject,
      you might find <link linkend="actors-composite">this recipe</link>
      (which goes into GObject in more detail) a useful introduction.</para>

      <example id="effects-custom-deform-example-1">
        <title><filename>cb-page-fold-effect.h</filename> (header file)</title>
        <programlisting>
<xi:include href="examples/cb-page-fold-effect.h" parse="text">
  <xi:fallback>a code sample should be here... but isn't</xi:fallback>
</xi:include>
        </programlisting>
      </example>

      <example id="effects-custom-deform-example-2">
        <title><filename>cb-page-fold-effect.c</filename> (code file)</title>
        <programlisting>
<xi:include href="examples/cb-page-fold-effect.c" parse="text">
  <xi:fallback>a code sample should be here... but isn't</xi:fallback>
</xi:include>
        </programlisting>
      </example>

      <example id="effects-custom-deform-example-3">
        <title>Application which uses <type>CbPageFoldEffect</type>
        to do animated folding of a <type>ClutterTexture</type></title>
        <programlisting>
<xi:include href="examples/effects-custom-deform.c" parse="text">
  <xi:fallback>a code sample should be here... but isn't</xi:fallback>
</xi:include>
        </programlisting>
      </example>

    </section>

  </section>

</chapter>