isa/mutter
1
0
Fork 0
mirror of https://github.com/brl/mutter.git synced 2024-12-23 19:42:05 +00:00
Code Issues Packages Projects Releases Wiki Activity
dd7b1d53db
mutter/cogl/cogl-pipeline-hash-table.c

228 lines
6.9 KiB
C
Raw Normal View History

pipeline-cache: Use a shared hash table wrapper The pipeline cache contains three separate hash tables, one for the state affecting the vertex shaders, one for the fragment shaders and one for the resulting combined program. Previously these hash tables had a fair bit of duplicated code to calculate the hashes, check for equality and copy the pipeline when it is added. This patch moves the common bits of code to a new type called CoglPipelineHashTable which just wraps a GHashTable with a given set of state flags to use for hashing and checking for equality. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit 402796430c839038339e531363b8c2463f9b2a9e) Conflicts: cogl/Makefile.am
2013-03-25 16:54:51 +00:00
/*
* Cogl
*
* An object oriented GL/GLES Abstraction/Utility Layer
*
* Copyright (C) 2013 Intel Corporation.
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation; either
* version 2 of the License, or (at your option) any later version.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library. If not, see
* <http://www.gnu.org/licenses/>.
*
*
* Authors:
* Neil Roberts <neil@linux.intel.com>
* Robert Bragg <robert@linux.intel.com>
*/
#ifdef HAVE_CONFIG_H
#include "config.h"
#endif
#include "cogl-context-private.h"
#include "cogl-pipeline-private.h"
#include "cogl-pipeline-hash-table.h"
pipeline-cache: Prune old unused pipelines when the cache gets too big Previously when a pipeline is added to the cache it would never be removed. If the application is generating a lot of unique pipelines this can end up effectively leaking a large number of resources including the GL program objects. Arguably this isn't really a problem because if the application is generating that many unique pipelines then it is doing something wrong anyway. It also implies that it will be recompiling shaders very often so the cache leaking will likely be the least of the problems. This patch makes it keep track of which pipelines in the cache are in use. The cache now returns a struct representing the entry instead of directly returning the pipeline. This entry contains a usage counter which the pipeline backends can use to mark when there is a pipeline alive that is using the cache entry. When the hash table decides that it's a good time to prune some entries, it will make a list of all of the pipelines that are not in use and then remove the least recently used half of the pipelines. That way it is less likely to remove pipelines that the application is actually regenerating often even if they aren't in use all of the time. When the cache is pruned the hash table makes a note of how small the cache could be if it removed all of the unused pipelines. The hash table starts pruning when there are more entries than twice this minimum expected size. The idea is that if that case it hit then the hash table is more than half full of useless pipelines so the application is generating lots of redundant pipelines and it is a good time to remove them. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit c21aac22992bb7fef5a8d0913130b8245e67f2eb) Conflicts: cogl/driver/gl/cogl-pipeline-fragend-glsl.c cogl/driver/gl/cogl-pipeline-progend-glsl.c cogl/driver/gl/cogl-pipeline-vertend-glsl.c cogl/driver/gl/gl/cogl-pipeline-fragend-arbfp.c
2013-12-18 15:18:39 +00:00
#include "cogl-pipeline-cache.h"
pipeline-cache: Use a shared hash table wrapper The pipeline cache contains three separate hash tables, one for the state affecting the vertex shaders, one for the fragment shaders and one for the resulting combined program. Previously these hash tables had a fair bit of duplicated code to calculate the hashes, check for equality and copy the pipeline when it is added. This patch moves the common bits of code to a new type called CoglPipelineHashTable which just wraps a GHashTable with a given set of state flags to use for hashing and checking for equality. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit 402796430c839038339e531363b8c2463f9b2a9e) Conflicts: cogl/Makefile.am
2013-03-25 16:54:51 +00:00
typedef struct
{
pipeline-cache: Prune old unused pipelines when the cache gets too big Previously when a pipeline is added to the cache it would never be removed. If the application is generating a lot of unique pipelines this can end up effectively leaking a large number of resources including the GL program objects. Arguably this isn't really a problem because if the application is generating that many unique pipelines then it is doing something wrong anyway. It also implies that it will be recompiling shaders very often so the cache leaking will likely be the least of the problems. This patch makes it keep track of which pipelines in the cache are in use. The cache now returns a struct representing the entry instead of directly returning the pipeline. This entry contains a usage counter which the pipeline backends can use to mark when there is a pipeline alive that is using the cache entry. When the hash table decides that it's a good time to prune some entries, it will make a list of all of the pipelines that are not in use and then remove the least recently used half of the pipelines. That way it is less likely to remove pipelines that the application is actually regenerating often even if they aren't in use all of the time. When the cache is pruned the hash table makes a note of how small the cache could be if it removed all of the unused pipelines. The hash table starts pruning when there are more entries than twice this minimum expected size. The idea is that if that case it hit then the hash table is more than half full of useless pipelines so the application is generating lots of redundant pipelines and it is a good time to remove them. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit c21aac22992bb7fef5a8d0913130b8245e67f2eb) Conflicts: cogl/driver/gl/cogl-pipeline-fragend-glsl.c cogl/driver/gl/cogl-pipeline-progend-glsl.c cogl/driver/gl/cogl-pipeline-vertend-glsl.c cogl/driver/gl/gl/cogl-pipeline-fragend-arbfp.c
2013-12-18 15:18:39 +00:00
CoglPipelineCacheEntry parent;
pipeline-cache: Use a shared hash table wrapper The pipeline cache contains three separate hash tables, one for the state affecting the vertex shaders, one for the fragment shaders and one for the resulting combined program. Previously these hash tables had a fair bit of duplicated code to calculate the hashes, check for equality and copy the pipeline when it is added. This patch moves the common bits of code to a new type called CoglPipelineHashTable which just wraps a GHashTable with a given set of state flags to use for hashing and checking for equality. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit 402796430c839038339e531363b8c2463f9b2a9e) Conflicts: cogl/Makefile.am
2013-03-25 16:54:51 +00:00
pipeline-hash-table: Store the hash value in the entry Calculating the hash value for a pipeline can be a bit expensive. Previously when adding a new pipeline to the hash table we would end up calculating the hash value once when checking whether the pipeline is already in the hash table and then again when adding the pipeline to the hash table. Ideally GHashTable would provide some API to add an entry with a precalculated hash value to avoid the recalculation, but seeing as it doesn't do that we can force it to avoid recalculating by storing the hash value as part of the struct we are using for the key. That way the hash func passed to GHashTable can simply return the precalculated value and we can calculate the hash outside of the GHashTable calls. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit 4a0460a452fd1be382fd5a92d8cfd5e0cdfd4403)
2013-03-26 17:53:02 +00:00
/* Calculating the hash is a little bit expensive for pipelines so
* we don't want to do it repeatedly for entries that are already in
* the hash table. Instead we cache the value here and calculate it
* outside of the GHashTable. */
unsigned int hash_value;
pipeline-cache: Use a shared hash table wrapper The pipeline cache contains three separate hash tables, one for the state affecting the vertex shaders, one for the fragment shaders and one for the resulting combined program. Previously these hash tables had a fair bit of duplicated code to calculate the hashes, check for equality and copy the pipeline when it is added. This patch moves the common bits of code to a new type called CoglPipelineHashTable which just wraps a GHashTable with a given set of state flags to use for hashing and checking for equality. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit 402796430c839038339e531363b8c2463f9b2a9e) Conflicts: cogl/Makefile.am
2013-03-25 16:54:51 +00:00
/* GHashTable annoyingly doesn't let us pass a user data pointer to
* the hash and equal functions so to work around it we have to
* store the pointer in every hash table entry. We will use this
* entry as both the key and the value */
CoglPipelineHashTable *hash;
pipeline-cache: Prune old unused pipelines when the cache gets too big Previously when a pipeline is added to the cache it would never be removed. If the application is generating a lot of unique pipelines this can end up effectively leaking a large number of resources including the GL program objects. Arguably this isn't really a problem because if the application is generating that many unique pipelines then it is doing something wrong anyway. It also implies that it will be recompiling shaders very often so the cache leaking will likely be the least of the problems. This patch makes it keep track of which pipelines in the cache are in use. The cache now returns a struct representing the entry instead of directly returning the pipeline. This entry contains a usage counter which the pipeline backends can use to mark when there is a pipeline alive that is using the cache entry. When the hash table decides that it's a good time to prune some entries, it will make a list of all of the pipelines that are not in use and then remove the least recently used half of the pipelines. That way it is less likely to remove pipelines that the application is actually regenerating often even if they aren't in use all of the time. When the cache is pruned the hash table makes a note of how small the cache could be if it removed all of the unused pipelines. The hash table starts pruning when there are more entries than twice this minimum expected size. The idea is that if that case it hit then the hash table is more than half full of useless pipelines so the application is generating lots of redundant pipelines and it is a good time to remove them. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit c21aac22992bb7fef5a8d0913130b8245e67f2eb) Conflicts: cogl/driver/gl/cogl-pipeline-fragend-glsl.c cogl/driver/gl/cogl-pipeline-progend-glsl.c cogl/driver/gl/cogl-pipeline-vertend-glsl.c cogl/driver/gl/gl/cogl-pipeline-fragend-arbfp.c
2013-12-18 15:18:39 +00:00
/* The number of unique pipelines that had been created when this
* pipeline was last accessed */
int age;
pipeline-cache: Use a shared hash table wrapper The pipeline cache contains three separate hash tables, one for the state affecting the vertex shaders, one for the fragment shaders and one for the resulting combined program. Previously these hash tables had a fair bit of duplicated code to calculate the hashes, check for equality and copy the pipeline when it is added. This patch moves the common bits of code to a new type called CoglPipelineHashTable which just wraps a GHashTable with a given set of state flags to use for hashing and checking for equality. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit 402796430c839038339e531363b8c2463f9b2a9e) Conflicts: cogl/Makefile.am
2013-03-25 16:54:51 +00:00
} CoglPipelineHashTableEntry;
static void
value_destroy_cb (void *value)
{
CoglPipelineHashTableEntry *entry = value;
pipeline-cache: Prune old unused pipelines when the cache gets too big Previously when a pipeline is added to the cache it would never be removed. If the application is generating a lot of unique pipelines this can end up effectively leaking a large number of resources including the GL program objects. Arguably this isn't really a problem because if the application is generating that many unique pipelines then it is doing something wrong anyway. It also implies that it will be recompiling shaders very often so the cache leaking will likely be the least of the problems. This patch makes it keep track of which pipelines in the cache are in use. The cache now returns a struct representing the entry instead of directly returning the pipeline. This entry contains a usage counter which the pipeline backends can use to mark when there is a pipeline alive that is using the cache entry. When the hash table decides that it's a good time to prune some entries, it will make a list of all of the pipelines that are not in use and then remove the least recently used half of the pipelines. That way it is less likely to remove pipelines that the application is actually regenerating often even if they aren't in use all of the time. When the cache is pruned the hash table makes a note of how small the cache could be if it removed all of the unused pipelines. The hash table starts pruning when there are more entries than twice this minimum expected size. The idea is that if that case it hit then the hash table is more than half full of useless pipelines so the application is generating lots of redundant pipelines and it is a good time to remove them. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit c21aac22992bb7fef5a8d0913130b8245e67f2eb) Conflicts: cogl/driver/gl/cogl-pipeline-fragend-glsl.c cogl/driver/gl/cogl-pipeline-progend-glsl.c cogl/driver/gl/cogl-pipeline-vertend-glsl.c cogl/driver/gl/gl/cogl-pipeline-fragend-arbfp.c
2013-12-18 15:18:39 +00:00
cogl_object_unref (entry->parent.pipeline);
pipeline-cache: Use a shared hash table wrapper The pipeline cache contains three separate hash tables, one for the state affecting the vertex shaders, one for the fragment shaders and one for the resulting combined program. Previously these hash tables had a fair bit of duplicated code to calculate the hashes, check for equality and copy the pipeline when it is added. This patch moves the common bits of code to a new type called CoglPipelineHashTable which just wraps a GHashTable with a given set of state flags to use for hashing and checking for equality. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit 402796430c839038339e531363b8c2463f9b2a9e) Conflicts: cogl/Makefile.am
2013-03-25 16:54:51 +00:00
g_slice_free (CoglPipelineHashTableEntry, entry);
}
static unsigned int
entry_hash (const void *data)
{
const CoglPipelineHashTableEntry *entry = data;
pipeline-hash-table: Store the hash value in the entry Calculating the hash value for a pipeline can be a bit expensive. Previously when adding a new pipeline to the hash table we would end up calculating the hash value once when checking whether the pipeline is already in the hash table and then again when adding the pipeline to the hash table. Ideally GHashTable would provide some API to add an entry with a precalculated hash value to avoid the recalculation, but seeing as it doesn't do that we can force it to avoid recalculating by storing the hash value as part of the struct we are using for the key. That way the hash func passed to GHashTable can simply return the precalculated value and we can calculate the hash outside of the GHashTable calls. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit 4a0460a452fd1be382fd5a92d8cfd5e0cdfd4403)
2013-03-26 17:53:02 +00:00
return entry->hash_value;
pipeline-cache: Use a shared hash table wrapper The pipeline cache contains three separate hash tables, one for the state affecting the vertex shaders, one for the fragment shaders and one for the resulting combined program. Previously these hash tables had a fair bit of duplicated code to calculate the hashes, check for equality and copy the pipeline when it is added. This patch moves the common bits of code to a new type called CoglPipelineHashTable which just wraps a GHashTable with a given set of state flags to use for hashing and checking for equality. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit 402796430c839038339e531363b8c2463f9b2a9e) Conflicts: cogl/Makefile.am
2013-03-25 16:54:51 +00:00
}
static CoglBool
entry_equal (const void *a,
const void *b)
{
const CoglPipelineHashTableEntry *entry_a = a;
const CoglPipelineHashTableEntry *entry_b = b;
const CoglPipelineHashTable *hash = entry_a->hash;
pipeline-cache: Prune old unused pipelines when the cache gets too big Previously when a pipeline is added to the cache it would never be removed. If the application is generating a lot of unique pipelines this can end up effectively leaking a large number of resources including the GL program objects. Arguably this isn't really a problem because if the application is generating that many unique pipelines then it is doing something wrong anyway. It also implies that it will be recompiling shaders very often so the cache leaking will likely be the least of the problems. This patch makes it keep track of which pipelines in the cache are in use. The cache now returns a struct representing the entry instead of directly returning the pipeline. This entry contains a usage counter which the pipeline backends can use to mark when there is a pipeline alive that is using the cache entry. When the hash table decides that it's a good time to prune some entries, it will make a list of all of the pipelines that are not in use and then remove the least recently used half of the pipelines. That way it is less likely to remove pipelines that the application is actually regenerating often even if they aren't in use all of the time. When the cache is pruned the hash table makes a note of how small the cache could be if it removed all of the unused pipelines. The hash table starts pruning when there are more entries than twice this minimum expected size. The idea is that if that case it hit then the hash table is more than half full of useless pipelines so the application is generating lots of redundant pipelines and it is a good time to remove them. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit c21aac22992bb7fef5a8d0913130b8245e67f2eb) Conflicts: cogl/driver/gl/cogl-pipeline-fragend-glsl.c cogl/driver/gl/cogl-pipeline-progend-glsl.c cogl/driver/gl/cogl-pipeline-vertend-glsl.c cogl/driver/gl/gl/cogl-pipeline-fragend-arbfp.c
2013-12-18 15:18:39 +00:00
return _cogl_pipeline_equal (entry_a->parent.pipeline,
entry_b->parent.pipeline,
pipeline-cache: Use a shared hash table wrapper The pipeline cache contains three separate hash tables, one for the state affecting the vertex shaders, one for the fragment shaders and one for the resulting combined program. Previously these hash tables had a fair bit of duplicated code to calculate the hashes, check for equality and copy the pipeline when it is added. This patch moves the common bits of code to a new type called CoglPipelineHashTable which just wraps a GHashTable with a given set of state flags to use for hashing and checking for equality. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit 402796430c839038339e531363b8c2463f9b2a9e) Conflicts: cogl/Makefile.am
2013-03-25 16:54:51 +00:00
hash->main_state,
hash->layer_state,
0);
}
void
_cogl_pipeline_hash_table_init (CoglPipelineHashTable *hash,
unsigned int main_state,
unsigned int layer_state,
const char *debug_string)
{
hash->n_unique_pipelines = 0;
hash->debug_string = debug_string;
hash->main_state = main_state;
hash->layer_state = layer_state;
pipeline-cache: Prune old unused pipelines when the cache gets too big Previously when a pipeline is added to the cache it would never be removed. If the application is generating a lot of unique pipelines this can end up effectively leaking a large number of resources including the GL program objects. Arguably this isn't really a problem because if the application is generating that many unique pipelines then it is doing something wrong anyway. It also implies that it will be recompiling shaders very often so the cache leaking will likely be the least of the problems. This patch makes it keep track of which pipelines in the cache are in use. The cache now returns a struct representing the entry instead of directly returning the pipeline. This entry contains a usage counter which the pipeline backends can use to mark when there is a pipeline alive that is using the cache entry. When the hash table decides that it's a good time to prune some entries, it will make a list of all of the pipelines that are not in use and then remove the least recently used half of the pipelines. That way it is less likely to remove pipelines that the application is actually regenerating often even if they aren't in use all of the time. When the cache is pruned the hash table makes a note of how small the cache could be if it removed all of the unused pipelines. The hash table starts pruning when there are more entries than twice this minimum expected size. The idea is that if that case it hit then the hash table is more than half full of useless pipelines so the application is generating lots of redundant pipelines and it is a good time to remove them. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit c21aac22992bb7fef5a8d0913130b8245e67f2eb) Conflicts: cogl/driver/gl/cogl-pipeline-fragend-glsl.c cogl/driver/gl/cogl-pipeline-progend-glsl.c cogl/driver/gl/cogl-pipeline-vertend-glsl.c cogl/driver/gl/gl/cogl-pipeline-fragend-arbfp.c
2013-12-18 15:18:39 +00:00
/* We'll only start pruning once we get to 16 unique pipelines */
hash->expected_min_size = 8;
pipeline-cache: Use a shared hash table wrapper The pipeline cache contains three separate hash tables, one for the state affecting the vertex shaders, one for the fragment shaders and one for the resulting combined program. Previously these hash tables had a fair bit of duplicated code to calculate the hashes, check for equality and copy the pipeline when it is added. This patch moves the common bits of code to a new type called CoglPipelineHashTable which just wraps a GHashTable with a given set of state flags to use for hashing and checking for equality. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit 402796430c839038339e531363b8c2463f9b2a9e) Conflicts: cogl/Makefile.am
2013-03-25 16:54:51 +00:00
hash->table = g_hash_table_new_full (entry_hash,
entry_equal,
NULL, /* key destroy */
value_destroy_cb);
}
void
_cogl_pipeline_hash_table_destroy (CoglPipelineHashTable *hash)
{
g_hash_table_destroy (hash->table);
}
pipeline-cache: Prune old unused pipelines when the cache gets too big Previously when a pipeline is added to the cache it would never be removed. If the application is generating a lot of unique pipelines this can end up effectively leaking a large number of resources including the GL program objects. Arguably this isn't really a problem because if the application is generating that many unique pipelines then it is doing something wrong anyway. It also implies that it will be recompiling shaders very often so the cache leaking will likely be the least of the problems. This patch makes it keep track of which pipelines in the cache are in use. The cache now returns a struct representing the entry instead of directly returning the pipeline. This entry contains a usage counter which the pipeline backends can use to mark when there is a pipeline alive that is using the cache entry. When the hash table decides that it's a good time to prune some entries, it will make a list of all of the pipelines that are not in use and then remove the least recently used half of the pipelines. That way it is less likely to remove pipelines that the application is actually regenerating often even if they aren't in use all of the time. When the cache is pruned the hash table makes a note of how small the cache could be if it removed all of the unused pipelines. The hash table starts pruning when there are more entries than twice this minimum expected size. The idea is that if that case it hit then the hash table is more than half full of useless pipelines so the application is generating lots of redundant pipelines and it is a good time to remove them. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit c21aac22992bb7fef5a8d0913130b8245e67f2eb) Conflicts: cogl/driver/gl/cogl-pipeline-fragend-glsl.c cogl/driver/gl/cogl-pipeline-progend-glsl.c cogl/driver/gl/cogl-pipeline-vertend-glsl.c cogl/driver/gl/gl/cogl-pipeline-fragend-arbfp.c
2013-12-18 15:18:39 +00:00
static void
collect_prunable_entries_cb (void *key,
void *value,
void *user_data)
{
GQueue *entries = user_data;
CoglPipelineCacheEntry *entry = value;
if (entry->usage_count == 0)
g_queue_push_tail (entries, entry);
}
static int
compare_pipeline_age_cb (const void *a,
const void *b)
{
const CoglPipelineHashTableEntry *ae = a;
const CoglPipelineHashTableEntry *be = b;
return be->age - ae->age;
}
static void
prune_old_pipelines (CoglPipelineHashTable *hash)
{
GQueue entries;
GList *l;
int i;
/* Collect all of the prunable entries into a GQueue */
g_queue_init (&entries);
g_hash_table_foreach (hash->table,
collect_prunable_entries_cb,
&entries);
/* Sort the entries by increasing order of age */
entries.head = g_list_sort (entries.head, compare_pipeline_age_cb);
/* The +1 is to include the pipeline that we're about to add */
hash->expected_min_size = (g_hash_table_size (hash->table) -
entries.length +
1);
/* Remove oldest half of the prunable pipelines. We still want to
* keep some of the prunable entries that are recently used because
* it's not unlikely that the application will recreate the same
* pipeline */
for (l = entries.head, i = 0; i < entries.length / 2; l = l->next, i++)
{
CoglPipelineCacheEntry *entry = l->data;
g_hash_table_remove (hash->table, entry);
}
g_list_free (entries.head);
}
CoglPipelineCacheEntry *
pipeline-cache: Use a shared hash table wrapper The pipeline cache contains three separate hash tables, one for the state affecting the vertex shaders, one for the fragment shaders and one for the resulting combined program. Previously these hash tables had a fair bit of duplicated code to calculate the hashes, check for equality and copy the pipeline when it is added. This patch moves the common bits of code to a new type called CoglPipelineHashTable which just wraps a GHashTable with a given set of state flags to use for hashing and checking for equality. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit 402796430c839038339e531363b8c2463f9b2a9e) Conflicts: cogl/Makefile.am
2013-03-25 16:54:51 +00:00
_cogl_pipeline_hash_table_get (CoglPipelineHashTable *hash,
CoglPipeline *key_pipeline)
{
CoglPipelineHashTableEntry dummy_entry;
CoglPipelineHashTableEntry *entry;
unsigned int copy_state;
pipeline-cache: Prune old unused pipelines when the cache gets too big Previously when a pipeline is added to the cache it would never be removed. If the application is generating a lot of unique pipelines this can end up effectively leaking a large number of resources including the GL program objects. Arguably this isn't really a problem because if the application is generating that many unique pipelines then it is doing something wrong anyway. It also implies that it will be recompiling shaders very often so the cache leaking will likely be the least of the problems. This patch makes it keep track of which pipelines in the cache are in use. The cache now returns a struct representing the entry instead of directly returning the pipeline. This entry contains a usage counter which the pipeline backends can use to mark when there is a pipeline alive that is using the cache entry. When the hash table decides that it's a good time to prune some entries, it will make a list of all of the pipelines that are not in use and then remove the least recently used half of the pipelines. That way it is less likely to remove pipelines that the application is actually regenerating often even if they aren't in use all of the time. When the cache is pruned the hash table makes a note of how small the cache could be if it removed all of the unused pipelines. The hash table starts pruning when there are more entries than twice this minimum expected size. The idea is that if that case it hit then the hash table is more than half full of useless pipelines so the application is generating lots of redundant pipelines and it is a good time to remove them. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit c21aac22992bb7fef5a8d0913130b8245e67f2eb) Conflicts: cogl/driver/gl/cogl-pipeline-fragend-glsl.c cogl/driver/gl/cogl-pipeline-progend-glsl.c cogl/driver/gl/cogl-pipeline-vertend-glsl.c cogl/driver/gl/gl/cogl-pipeline-fragend-arbfp.c
2013-12-18 15:18:39 +00:00
dummy_entry.parent.pipeline = key_pipeline;
pipeline-cache: Use a shared hash table wrapper The pipeline cache contains three separate hash tables, one for the state affecting the vertex shaders, one for the fragment shaders and one for the resulting combined program. Previously these hash tables had a fair bit of duplicated code to calculate the hashes, check for equality and copy the pipeline when it is added. This patch moves the common bits of code to a new type called CoglPipelineHashTable which just wraps a GHashTable with a given set of state flags to use for hashing and checking for equality. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit 402796430c839038339e531363b8c2463f9b2a9e) Conflicts: cogl/Makefile.am
2013-03-25 16:54:51 +00:00
dummy_entry.hash = hash;
pipeline-hash-table: Store the hash value in the entry Calculating the hash value for a pipeline can be a bit expensive. Previously when adding a new pipeline to the hash table we would end up calculating the hash value once when checking whether the pipeline is already in the hash table and then again when adding the pipeline to the hash table. Ideally GHashTable would provide some API to add an entry with a precalculated hash value to avoid the recalculation, but seeing as it doesn't do that we can force it to avoid recalculating by storing the hash value as part of the struct we are using for the key. That way the hash func passed to GHashTable can simply return the precalculated value and we can calculate the hash outside of the GHashTable calls. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit 4a0460a452fd1be382fd5a92d8cfd5e0cdfd4403)
2013-03-26 17:53:02 +00:00
dummy_entry.hash_value = _cogl_pipeline_hash (key_pipeline,
hash->main_state,
hash->layer_state,
0);
pipeline-cache: Use a shared hash table wrapper The pipeline cache contains three separate hash tables, one for the state affecting the vertex shaders, one for the fragment shaders and one for the resulting combined program. Previously these hash tables had a fair bit of duplicated code to calculate the hashes, check for equality and copy the pipeline when it is added. This patch moves the common bits of code to a new type called CoglPipelineHashTable which just wraps a GHashTable with a given set of state flags to use for hashing and checking for equality. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit 402796430c839038339e531363b8c2463f9b2a9e) Conflicts: cogl/Makefile.am
2013-03-25 16:54:51 +00:00
entry = g_hash_table_lookup (hash->table, &dummy_entry);
if (entry)
pipeline-cache: Prune old unused pipelines when the cache gets too big Previously when a pipeline is added to the cache it would never be removed. If the application is generating a lot of unique pipelines this can end up effectively leaking a large number of resources including the GL program objects. Arguably this isn't really a problem because if the application is generating that many unique pipelines then it is doing something wrong anyway. It also implies that it will be recompiling shaders very often so the cache leaking will likely be the least of the problems. This patch makes it keep track of which pipelines in the cache are in use. The cache now returns a struct representing the entry instead of directly returning the pipeline. This entry contains a usage counter which the pipeline backends can use to mark when there is a pipeline alive that is using the cache entry. When the hash table decides that it's a good time to prune some entries, it will make a list of all of the pipelines that are not in use and then remove the least recently used half of the pipelines. That way it is less likely to remove pipelines that the application is actually regenerating often even if they aren't in use all of the time. When the cache is pruned the hash table makes a note of how small the cache could be if it removed all of the unused pipelines. The hash table starts pruning when there are more entries than twice this minimum expected size. The idea is that if that case it hit then the hash table is more than half full of useless pipelines so the application is generating lots of redundant pipelines and it is a good time to remove them. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit c21aac22992bb7fef5a8d0913130b8245e67f2eb) Conflicts: cogl/driver/gl/cogl-pipeline-fragend-glsl.c cogl/driver/gl/cogl-pipeline-progend-glsl.c cogl/driver/gl/cogl-pipeline-vertend-glsl.c cogl/driver/gl/gl/cogl-pipeline-fragend-arbfp.c
2013-12-18 15:18:39 +00:00
{
entry->age = hash->n_unique_pipelines;
return &entry->parent;
}
pipeline-cache: Use a shared hash table wrapper The pipeline cache contains three separate hash tables, one for the state affecting the vertex shaders, one for the fragment shaders and one for the resulting combined program. Previously these hash tables had a fair bit of duplicated code to calculate the hashes, check for equality and copy the pipeline when it is added. This patch moves the common bits of code to a new type called CoglPipelineHashTable which just wraps a GHashTable with a given set of state flags to use for hashing and checking for equality. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit 402796430c839038339e531363b8c2463f9b2a9e) Conflicts: cogl/Makefile.am
2013-03-25 16:54:51 +00:00
if (hash->n_unique_pipelines == 50)
g_warning ("Over 50 separate %s have been generated which is very "
"unusual, so something is probably wrong!\n",
hash->debug_string);
pipeline-cache: Prune old unused pipelines when the cache gets too big Previously when a pipeline is added to the cache it would never be removed. If the application is generating a lot of unique pipelines this can end up effectively leaking a large number of resources including the GL program objects. Arguably this isn't really a problem because if the application is generating that many unique pipelines then it is doing something wrong anyway. It also implies that it will be recompiling shaders very often so the cache leaking will likely be the least of the problems. This patch makes it keep track of which pipelines in the cache are in use. The cache now returns a struct representing the entry instead of directly returning the pipeline. This entry contains a usage counter which the pipeline backends can use to mark when there is a pipeline alive that is using the cache entry. When the hash table decides that it's a good time to prune some entries, it will make a list of all of the pipelines that are not in use and then remove the least recently used half of the pipelines. That way it is less likely to remove pipelines that the application is actually regenerating often even if they aren't in use all of the time. When the cache is pruned the hash table makes a note of how small the cache could be if it removed all of the unused pipelines. The hash table starts pruning when there are more entries than twice this minimum expected size. The idea is that if that case it hit then the hash table is more than half full of useless pipelines so the application is generating lots of redundant pipelines and it is a good time to remove them. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit c21aac22992bb7fef5a8d0913130b8245e67f2eb) Conflicts: cogl/driver/gl/cogl-pipeline-fragend-glsl.c cogl/driver/gl/cogl-pipeline-progend-glsl.c cogl/driver/gl/cogl-pipeline-vertend-glsl.c cogl/driver/gl/gl/cogl-pipeline-fragend-arbfp.c
2013-12-18 15:18:39 +00:00
/* If we are going to have more than twice the expected minimum
* number of pipelines in the hash then we'll try pruning and update
* the minimum */
if (g_hash_table_size (hash->table) >= hash->expected_min_size * 2)
prune_old_pipelines (hash);
pipeline-cache: Use a shared hash table wrapper The pipeline cache contains three separate hash tables, one for the state affecting the vertex shaders, one for the fragment shaders and one for the resulting combined program. Previously these hash tables had a fair bit of duplicated code to calculate the hashes, check for equality and copy the pipeline when it is added. This patch moves the common bits of code to a new type called CoglPipelineHashTable which just wraps a GHashTable with a given set of state flags to use for hashing and checking for equality. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit 402796430c839038339e531363b8c2463f9b2a9e) Conflicts: cogl/Makefile.am
2013-03-25 16:54:51 +00:00
entry = g_slice_new (CoglPipelineHashTableEntry);
pipeline-cache: Prune old unused pipelines when the cache gets too big Previously when a pipeline is added to the cache it would never be removed. If the application is generating a lot of unique pipelines this can end up effectively leaking a large number of resources including the GL program objects. Arguably this isn't really a problem because if the application is generating that many unique pipelines then it is doing something wrong anyway. It also implies that it will be recompiling shaders very often so the cache leaking will likely be the least of the problems. This patch makes it keep track of which pipelines in the cache are in use. The cache now returns a struct representing the entry instead of directly returning the pipeline. This entry contains a usage counter which the pipeline backends can use to mark when there is a pipeline alive that is using the cache entry. When the hash table decides that it's a good time to prune some entries, it will make a list of all of the pipelines that are not in use and then remove the least recently used half of the pipelines. That way it is less likely to remove pipelines that the application is actually regenerating often even if they aren't in use all of the time. When the cache is pruned the hash table makes a note of how small the cache could be if it removed all of the unused pipelines. The hash table starts pruning when there are more entries than twice this minimum expected size. The idea is that if that case it hit then the hash table is more than half full of useless pipelines so the application is generating lots of redundant pipelines and it is a good time to remove them. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit c21aac22992bb7fef5a8d0913130b8245e67f2eb) Conflicts: cogl/driver/gl/cogl-pipeline-fragend-glsl.c cogl/driver/gl/cogl-pipeline-progend-glsl.c cogl/driver/gl/cogl-pipeline-vertend-glsl.c cogl/driver/gl/gl/cogl-pipeline-fragend-arbfp.c
2013-12-18 15:18:39 +00:00
entry->parent.usage_count = 0;
pipeline-cache: Use a shared hash table wrapper The pipeline cache contains three separate hash tables, one for the state affecting the vertex shaders, one for the fragment shaders and one for the resulting combined program. Previously these hash tables had a fair bit of duplicated code to calculate the hashes, check for equality and copy the pipeline when it is added. This patch moves the common bits of code to a new type called CoglPipelineHashTable which just wraps a GHashTable with a given set of state flags to use for hashing and checking for equality. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit 402796430c839038339e531363b8c2463f9b2a9e) Conflicts: cogl/Makefile.am
2013-03-25 16:54:51 +00:00
entry->hash = hash;
pipeline-hash-table: Store the hash value in the entry Calculating the hash value for a pipeline can be a bit expensive. Previously when adding a new pipeline to the hash table we would end up calculating the hash value once when checking whether the pipeline is already in the hash table and then again when adding the pipeline to the hash table. Ideally GHashTable would provide some API to add an entry with a precalculated hash value to avoid the recalculation, but seeing as it doesn't do that we can force it to avoid recalculating by storing the hash value as part of the struct we are using for the key. That way the hash func passed to GHashTable can simply return the precalculated value and we can calculate the hash outside of the GHashTable calls. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit 4a0460a452fd1be382fd5a92d8cfd5e0cdfd4403)
2013-03-26 17:53:02 +00:00
entry->hash_value = dummy_entry.hash_value;
pipeline-cache: Prune old unused pipelines when the cache gets too big Previously when a pipeline is added to the cache it would never be removed. If the application is generating a lot of unique pipelines this can end up effectively leaking a large number of resources including the GL program objects. Arguably this isn't really a problem because if the application is generating that many unique pipelines then it is doing something wrong anyway. It also implies that it will be recompiling shaders very often so the cache leaking will likely be the least of the problems. This patch makes it keep track of which pipelines in the cache are in use. The cache now returns a struct representing the entry instead of directly returning the pipeline. This entry contains a usage counter which the pipeline backends can use to mark when there is a pipeline alive that is using the cache entry. When the hash table decides that it's a good time to prune some entries, it will make a list of all of the pipelines that are not in use and then remove the least recently used half of the pipelines. That way it is less likely to remove pipelines that the application is actually regenerating often even if they aren't in use all of the time. When the cache is pruned the hash table makes a note of how small the cache could be if it removed all of the unused pipelines. The hash table starts pruning when there are more entries than twice this minimum expected size. The idea is that if that case it hit then the hash table is more than half full of useless pipelines so the application is generating lots of redundant pipelines and it is a good time to remove them. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit c21aac22992bb7fef5a8d0913130b8245e67f2eb) Conflicts: cogl/driver/gl/cogl-pipeline-fragend-glsl.c cogl/driver/gl/cogl-pipeline-progend-glsl.c cogl/driver/gl/cogl-pipeline-vertend-glsl.c cogl/driver/gl/gl/cogl-pipeline-fragend-arbfp.c
2013-12-18 15:18:39 +00:00
entry->age = hash->n_unique_pipelines;
pipeline-cache: Use a shared hash table wrapper The pipeline cache contains three separate hash tables, one for the state affecting the vertex shaders, one for the fragment shaders and one for the resulting combined program. Previously these hash tables had a fair bit of duplicated code to calculate the hashes, check for equality and copy the pipeline when it is added. This patch moves the common bits of code to a new type called CoglPipelineHashTable which just wraps a GHashTable with a given set of state flags to use for hashing and checking for equality. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit 402796430c839038339e531363b8c2463f9b2a9e) Conflicts: cogl/Makefile.am
2013-03-25 16:54:51 +00:00
pipeline-cache: Use a special trimmed down pipeline for the key When a pipeline is added to the cache, a normal copy would previously be made to use as the key in the hash table. This copy keeps a reference to the real pipeline which means all of the resources it contains are retained forever, even if they aren't necessary to generate the hash. This patch changes it to create a trimmed down copy that only has the state necessary to generate the hash. A new function called _cogl_pipeline_deep_copy is added which makes a new pipeline that is directly a child of the root pipeline. It then copies over the pertinent state from the original pipeline. The pipeline state is copied using the existing _cogl_pipeline_copy_differences function. There was no equivalent function for the layer state so I have added one. That way the pipeline key doesn't have the texture data state and it doesn't hold a reference to the original pipeline so it should be much cheaper to keep around. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit e27e01c1215e7e7c7c0183ded11dd769bb112c5c)
2013-03-26 11:17:55 +00:00
copy_state = hash->main_state;
if (hash->layer_state)
copy_state |= COGL_PIPELINE_STATE_LAYERS;
/* Create a new pipeline that is a child of the root pipeline
* instead of a normal copy so that the template pipeline won't hold
* a reference to the original pipeline */
pipeline-cache: Prune old unused pipelines when the cache gets too big Previously when a pipeline is added to the cache it would never be removed. If the application is generating a lot of unique pipelines this can end up effectively leaking a large number of resources including the GL program objects. Arguably this isn't really a problem because if the application is generating that many unique pipelines then it is doing something wrong anyway. It also implies that it will be recompiling shaders very often so the cache leaking will likely be the least of the problems. This patch makes it keep track of which pipelines in the cache are in use. The cache now returns a struct representing the entry instead of directly returning the pipeline. This entry contains a usage counter which the pipeline backends can use to mark when there is a pipeline alive that is using the cache entry. When the hash table decides that it's a good time to prune some entries, it will make a list of all of the pipelines that are not in use and then remove the least recently used half of the pipelines. That way it is less likely to remove pipelines that the application is actually regenerating often even if they aren't in use all of the time. When the cache is pruned the hash table makes a note of how small the cache could be if it removed all of the unused pipelines. The hash table starts pruning when there are more entries than twice this minimum expected size. The idea is that if that case it hit then the hash table is more than half full of useless pipelines so the application is generating lots of redundant pipelines and it is a good time to remove them. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit c21aac22992bb7fef5a8d0913130b8245e67f2eb) Conflicts: cogl/driver/gl/cogl-pipeline-fragend-glsl.c cogl/driver/gl/cogl-pipeline-progend-glsl.c cogl/driver/gl/cogl-pipeline-vertend-glsl.c cogl/driver/gl/gl/cogl-pipeline-fragend-arbfp.c
2013-12-18 15:18:39 +00:00
entry->parent.pipeline = _cogl_pipeline_deep_copy (key_pipeline,
copy_state,
hash->layer_state);
pipeline-cache: Use a special trimmed down pipeline for the key When a pipeline is added to the cache, a normal copy would previously be made to use as the key in the hash table. This copy keeps a reference to the real pipeline which means all of the resources it contains are retained forever, even if they aren't necessary to generate the hash. This patch changes it to create a trimmed down copy that only has the state necessary to generate the hash. A new function called _cogl_pipeline_deep_copy is added which makes a new pipeline that is directly a child of the root pipeline. It then copies over the pertinent state from the original pipeline. The pipeline state is copied using the existing _cogl_pipeline_copy_differences function. There was no equivalent function for the layer state so I have added one. That way the pipeline key doesn't have the texture data state and it doesn't hold a reference to the original pipeline so it should be much cheaper to keep around. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit e27e01c1215e7e7c7c0183ded11dd769bb112c5c)
2013-03-26 11:17:55 +00:00
pipeline-cache: Use a shared hash table wrapper The pipeline cache contains three separate hash tables, one for the state affecting the vertex shaders, one for the fragment shaders and one for the resulting combined program. Previously these hash tables had a fair bit of duplicated code to calculate the hashes, check for equality and copy the pipeline when it is added. This patch moves the common bits of code to a new type called CoglPipelineHashTable which just wraps a GHashTable with a given set of state flags to use for hashing and checking for equality. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit 402796430c839038339e531363b8c2463f9b2a9e) Conflicts: cogl/Makefile.am
2013-03-25 16:54:51 +00:00
g_hash_table_insert (hash->table, entry, entry);
hash->n_unique_pipelines++;
pipeline-cache: Prune old unused pipelines when the cache gets too big Previously when a pipeline is added to the cache it would never be removed. If the application is generating a lot of unique pipelines this can end up effectively leaking a large number of resources including the GL program objects. Arguably this isn't really a problem because if the application is generating that many unique pipelines then it is doing something wrong anyway. It also implies that it will be recompiling shaders very often so the cache leaking will likely be the least of the problems. This patch makes it keep track of which pipelines in the cache are in use. The cache now returns a struct representing the entry instead of directly returning the pipeline. This entry contains a usage counter which the pipeline backends can use to mark when there is a pipeline alive that is using the cache entry. When the hash table decides that it's a good time to prune some entries, it will make a list of all of the pipelines that are not in use and then remove the least recently used half of the pipelines. That way it is less likely to remove pipelines that the application is actually regenerating often even if they aren't in use all of the time. When the cache is pruned the hash table makes a note of how small the cache could be if it removed all of the unused pipelines. The hash table starts pruning when there are more entries than twice this minimum expected size. The idea is that if that case it hit then the hash table is more than half full of useless pipelines so the application is generating lots of redundant pipelines and it is a good time to remove them. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit c21aac22992bb7fef5a8d0913130b8245e67f2eb) Conflicts: cogl/driver/gl/cogl-pipeline-fragend-glsl.c cogl/driver/gl/cogl-pipeline-progend-glsl.c cogl/driver/gl/cogl-pipeline-vertend-glsl.c cogl/driver/gl/gl/cogl-pipeline-fragend-arbfp.c
2013-12-18 15:18:39 +00:00
return &entry->parent;
pipeline-cache: Use a shared hash table wrapper The pipeline cache contains three separate hash tables, one for the state affecting the vertex shaders, one for the fragment shaders and one for the resulting combined program. Previously these hash tables had a fair bit of duplicated code to calculate the hashes, check for equality and copy the pipeline when it is added. This patch moves the common bits of code to a new type called CoglPipelineHashTable which just wraps a GHashTable with a given set of state flags to use for hashing and checking for equality. Reviewed-by: Robert Bragg <robert@linux.intel.com> (cherry picked from commit 402796430c839038339e531363b8c2463f9b2a9e) Conflicts: cogl/Makefile.am
2013-03-25 16:54:51 +00:00
}
Reference in New Issue Copy Permalink