A general approach for Stardust database access tuning consists of the following tasks:
First, you have to identify
The appropriate means for this task in general are database specific tools. Additionally, you may set the Stardust server property:
Carnot.Engine.Tuning.DB.slowStatementTracingThreshold = 500.
This property sets the threshold duration in milliseconds for all database statements launched by Stardust . If the threshold is exceeded, the corresponding statement is logged in the Stardust log. with WARN priority, e.g.
[06.12.05 12:01:53:443] [ORB.thread.pool : 4] [ WARN] Slow
query: 1085ms SELECT COUNT(*) FROM activity_instance o INNER JOIN
process_instance PR_PI ON (o.processInstance = PR_PI.oid) INNER JOIN data_value
PR_DV1 ON (PR_PI.rootProcessInstance = PR_DV1.processInstance AND PR_DV1.data =
?) INNER JOIN ikr_dcluster_1 PR_DVCL1 ON (PR_PI.rootProcessInstance =
PR_DVCL1.processInstance) WHERE (o.state IN (?, ?) AND
(PR_DVCL1.tk_an_bereich_bs = ? AND PR_DVCL1.sval_an_bereich_bs IN (?, ?, ?, ?,
?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
?, ?, ?, ?, ?, ?, ?, ?)) AND (PR_DV1.type_key = ? AND PR_DV1.string_value = ?)
AND (PR_DVCL1.tk_an_rolle = ? AND PR_DVCL1.sval_an_rolle = ?) AND
(o.currentUserPerformer = ? OR o.currentPerformer = ?))
A useful approach is to identify categories of critical statements with:
<session-name>.UsePreparedStatements = true
and to then deactivate prepared statement to obtain the concrete parameter values for critical queries (data IDs, process OIDs).
Execution plans for critical queries are obtained with the tools provided by the DBMS in use. You may consult your database administrator for further details.
The obtained list of critical statements can be used in two ways:
(A) A statement with multiple joins on the DATA_VALUE table -
especially if launched very often - is a good candidate for a cluster slot.
Data being used together in predicates
... WHERE ... Data1=? AND Data2=?
should reside in the same cluster.
Additionally, indexes on these clusters might be considered according to the following principles:
of the index definition.
(B) With or without clusters being defined, the execution path (table scans, index scans) has to be considered in detail and known database tuning mechanisms, like
on cluster and non-cluster tables have to be applied. To
decide on the appropriate approach you have to obtain the UPDATE/SELECT
ratio for the tables to be indexed. In any case, theses considerations are not
Stardust -specific and can be recommended by your database administrator
without further knowledge of the Stardust Process Engine.
The steps described below can be performed repeatedly with successively reducing the threshold for critical statements.
UPDATE-Deadlocks on data clusters caused by parallel access of two different threads on disjoint cluster slots can be avoided by splitting up the cluster. Critical clusters can be obtained from the fields listed in the corresponding log.
It might also be an approach to change the process model accordingly.
Deadlocks on other Stardust tables can be avoided by specifying additional indexes on these tables. This approach is also a standard database administration task and does not require specific knowledge of the Stardust products. Hence, you might consult your database administrator for further recommendations.
Per default, database generated keys are retrieved on non-sequence
databases after the INSERT statement. To avoid this extra round trip, you can
use the property AuditTrail.UseJdbcReturnGeneratedKeys. If you
add this property to the carnot.properties file and set it to true,
the support provided by JDBC 1.4 for retrieving database generated keys directly
from the INSERT statement will be used instead.
An audit trail database containing data for a large number of processes may reach a size, where the scalability limits of the used database system are reached. Usually, a high percentage of the process instances, whose data are stored in the audit trail database are already terminated. Hence, these process data can be exported to off line data sources and access only on demand.
To increase performance, you have the option to remove process instance
hierarchy entries upon process termination. If you set the server-side
property Carnot.Engine.Tuning.ProcessTerminationHierarchyCleanup
in your carnot.properties file to true, process instance
hierarchy entries are removed from the table procinst_hierarchy
upon process termination. Per default this property is set to false.
For details on table procinst_hierarchy, refer to the
PROCINST_HIERARCHY TABLE description in chapter
The Repository Model.
To reduce the number of statements the database needs to parse and optimize you can use prepared statements. In general, any query that is meant to be used more than once should be a prepared statement.
Using Prepared SQL Statements is usually estimated to raise performance by about 10%, no additional action needed. This has to be considered of special importance for complex queries like the ones issued from the Query API implementation. Now all queries and updates on the audit trail honor the session property to use Prepared SQL Statements:
<session-name>.UsePreparedStatements = false/true/mixed
Note that value false should only be used for debugging purposes! In production
scenarios please use true or mixed instead.
We recommend to use the mixed mode. For example, if querying for process instances states and activity instance states, the data can be "unbalanced". Mostly there are a lot of completed and only a few non-completed states. Regarding the performance, in case such data occur, it is helpful for the database query analyzer and data access path builder to have the real values for these unbalanced data and additionally the prepared statements for general business data.
Note that the mixed mode is deprecated. In a later release,
mixed SQL will be done by the engine internally if the property is true.
If prepared statements are used, the corresponding SQL queries will
contain bound values like ? characters instead of concrete values.
Note that prepared statements are explicitly ignored for the following scenarios:
ignorePreparedStatements=true and a query uses this slot.daemon_log.typedaemon_log.codedaemon_log.stateactivity_instance.stateactivity_instance.currentUserPerformeract_inst_history.stateact_inst_history.performerKindact_inst_history.onBehalfOfKindactivity_inst_log.typestructured_data_value.typeparticipant.typeevent_binding.typelog_entry.typelog_entry.codeprocess_instance.stateprocess_instance.priorityworkitem.stateact_inst_property.type_keyproc_inst_property.type_keyusergroup_property.type_keyuser_property.type_key Adaptive Cursor Sharing can be used for optimization as it allows the server to compare the effectiveness of execution plans between executions with different bind variable values. Please refer to Oracle's online documentation for details on the Oracle cursor sharing option, e.g. Improving Real-World Performance Through Cursor Sharing of the Oracle Tuning guide.
Using Oracle's adaptive cursor sharing requires a histogram creation. For example use
the following commands to force a histogram creation on column STRING_VALUE
of table DATA_VALUE:
EXEC DBMS_STATS.gather_table_stats(NULL, 'DATA_VALUE', method_opt=>'FOR COLUMNS SIZE 254 STRING_VALUE', cascade=>TRUE); SELECT column_name, histogram FROM user_tab_cols WHERE table_name = 'DATA_VALUE';
Note that the table must not be empty. It has to contain values to force histogram creation.
The following table shows the created histogram for column STRING_VALUE:
COLUMN_NAME HISTOGRAM ------------------------------ --------------- OID NONE MODEL NONE DATA NONE STRING_VALUE HEIGHT BALANCED NUMBER_VALUE NONE DOUBLE_VALUE NONE TYPE_KEY NONE PROCESSINSTANCE NONE
If the current schema is different from the schema the histogram table should be created in, use the following syntax:
EXEC DBMS_STATS.gather_table_stats('CARNOT', 'DATA_VALUE', method_opt=>'FOR COLUMNS SIZE 254 STRING_VALUE', cascade=>TRUE);
SELECT column_name, histogram FROM ALL_TAB_COLUMNS WHERE owner='CARNOT' AND table_name = 'DATA_VALUE';
In the first example, we run a query that uses a bind variable. The table DATA_VALUE has one entry that matches
data% for string_value. The execution plan shows that the query has used the index.
SQL> VARIABLE l_string_value VARCHAR2(128 CHAR);
SQL> EXEC :l_string_value := 'data%';
PL/SQL procedure successfully completed.
SQL>
SQL> SELECT MAX(oid) FROM carnot.data_value WHERE string_value LIKE :l_string_value;
MAX(OID)
----------
100006
SQL> SELECT * FROM TABLE(DBMS_XPLAN.display_cursor);
PLAN_TABLE_OUTPUT
---------------------------------------------------------------------------------------------------------
SQL_ID gzms5fbsmqwmp, child number 1
-------------------------------------
SELECT MAX(oid) FROM carnot.data_value WHERE string_value LIKE
:l_string_value
Plan hash value: 1688810086
---------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
---------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | | | 3 (100)| |
PLAN_TABLE_OUTPUT
---------------------------------------------------------------------------------------------------------
| 1 | SORT AGGREGATE | | 1 | 23 | | |
| 2 | TABLE ACCESS BY INDEX ROWID| DATA_VALUE | 1 | 23 | 3 (0)| 00:00:01 |
|* 3 | INDEX RANGE SCAN | DATA_VALUES_INDEX5 | 1 | | 2 (0)| 00:00:01 |
---------------------------------------------------------------------------------------------------
Now we run a query that uses a bind variable containing a value that matches multiple entries. The optimizer selects an execution plan that uses full table access.
VARIABLE l_string_value VARCHAR2(128 CHAR);
EXEC :l_string_value := '%test2%';
SELECT MAX(oid) FROM carnot.data_value WHERE string_value LIKE :l_string_value;
SELECT * FROM TABLE(DBMS_XPLAN.display_cursor);
SQL> VARIABLE l_string_value VARCHAR2(128 CHAR);
SQL> EXEC :l_string_value := '%test2%';
PL/SQL procedure successfully completed.
SQL>
SQL> SELECT MAX(oid) FROM carnot.data_value WHERE string_value LIKE :l_string_value;
MAX(OID)
----------
99986
SQL> SELECT * FROM TABLE(DBMS_XPLAN.display_cursor);
PLAN_TABLE_OUTPUT
----------------------------------------------------------------------------------------------------
SQL_ID gzms5fbsmqwmp, child number 3
-------------------------------------
SELECT MAX(oid) FROM carnot.data_value WHERE string_value LIKE
:l_string_value
Plan hash value: 3200669370
---------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
---------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | | | 141 (100)| |
PLAN_TABLE_OUTPUT
----------------------------------------------------------------------------------------------------
| 1 | SORT AGGREGATE | | 1 | 23 | | |
|* 2 | TABLE ACCESS FULL| DATA_VALUE | 5000 | 112K| 141 (2)| 00:00:02 |
---------------------------------------------------------------------------------
Now we run the following query:
SQL> SELECT sql_id, child_number, is_bind_sensitive, is_bind_aware 2 FROM v$sql 3 WHERE sql_text = 'SELECT MAX(oid) FROM carnot.data_value WHERE string_value LIKE :l_string_value';
Having a look a the output, we can see that, as column is_bind_sensitive displays
value Y, this query requires different execution plans depending on the bind variable value.
Column is_bind_aware shows that a more efficient execution plan for this query is used.
SQL_ID CHILD_NUMBER I I ------------- ------------ - - gzms5fbsmqwmp 0 Y N gzms5fbsmqwmp 1 Y Y gzms5fbsmqwmp 2 Y Y gzms5fbsmqwmp 3 Y Y
You can optimize the usage of prepared statements with a DB2 profile, which is described in this section. With this setup, a new execution plan is created for each executed statement specified in the profile.
Please refer to the IBM Knowledge Center site Using the REOPT bind option with input variables in complex queries and to article Improve SQL execution times using the DB2 statement concentrator feature and the DB2 statement reoptimization feature for details on using the DB2 statement re-optimization feature.
To create and setup a profile for optimization:
profile1.xml, with the following content:
<?xml version="1.0" encoding="UTF-8"?>
<OPTPROFILE VERSION="9.7.2.0">
<STMTPROFILE ID="REOPT_QUERY">
<STMTKEY>
<![CDATA[SELECT MAX(oid) FROM data_value WHERE string_value LIKE ?]]>
</STMTKEY>
<OPTGUIDELINES>
<REOPT VALUE="ALWAYS"/>
</OPTGUIDELINES>
</STMTPROFILE>
</OPTPROFILE>
DB2_OPTPROFILE
has to be set to YES. After the change the DB2 server has to be restarted:
db2set DB2_OPTPROFILE=YES db2stop db2start
SYSPROC.SYSINSTALLOBJECTS procedure to create the SYSTOOLS.OPT_PROFILE
table for storing the optimization profile:
db2 CALL SYSPROC.SYSINSTALLOBJECTS('OPT_PROFILES', 'C', CAST(NULL AS VARCHAR(128)), CAST(NULL AS VARCHAR(128)))profile1.del, with the following content:
"DEMO","PROFILE1","profile1.xml"
SYSTOOLS.OPT_PROFILE:
db2 IMPORT FROM profile1.del OF DEL MODIFIED BY LOBSINFILE INSERT INTO SYSTOOLS.OPT_PROFILE
db2 FLUSH OPTIMIZATION PROFILE CACHE ALL;
optimizationProfile:
jdbc:db2://localhost:50000/ipp:optimizationProfile=DEMO.PROFILE1;