# GNOME Data Access 6 manual

for version &version;
Michael
Lausch

    michael.lausch@1012surf.net

Rodrigo
Moya

    rodrigo@gnome-db.org

Sean
Allen

    zeroone@worldonline.co.za

GDP compliance, FDL, added mark-up, English and syntax
Xabier
Rodriguez Calvar
&igalia;

    xrcalvar@igalia.com

How to begin and migration formulae
Jose
Dapena Paz
&igalia;

    jdapena@igalia.com

Some examples
Bas
Driessen
&xobas;

    bas.driessen@xobas.com

libgda, gda-postgres, gda-mysql
Daniel
Espinosa Ortiz

    esodan@gmail.com

libgda, Vala, GObject Introspection
Pavlo
Solntsev

    p.sun.fun@gmail.com

libgda
Vivien
Malerba

    malerba@gnome-db.org

Current maintainer
&builddate;
1999 - 2018
&copy;

GNOME Data Access (&GDA;) is an architecture whose purpose is to provide universal access to many different kinds and types of data sources. This goes from traditional relational database systems, to any imaginable kind of data source such as a mail server, a &LDAP; directory...

This universality is obtained through the use of an easily extensible plug-in system as the mechanism for communication between the different components in the architecture.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license can be found in the appendix.

Many of the names used by companies to distinguish their products and services are claimed as trademarks. Where those names appear in any GNOME documentation, and those trademarks are made aware to the members of the GNOME Documentation Project, the names have been printed in caps or initial caps.

## Getting started

### Introduction

&ODBC; and &SQL; are established standards. However &ODBC; is mostly limited to the Microsoft Windows environment (even though the UnixODBC project exists and links to some drivers), and the API is quite old and not well integrated into the GNOME ecosystem.

&SQL; itself is standardized but up to some point only, so that &SQL; source compatibility can not be assured for all database servers. And for some sort of servers, &SQL; is not even feasible (think about &LDAP;).

&JDBC; is a more recent standard which replaces &ODBC; in the Java world, but it's limited to the Java programming world, and is difficult to integrate in C (or other languages different from Java). There are however a lot of JDBC drivers out there for many database engines.

&GDA; (GNOME Data Access) tries to tackle the &ODBC; and &JDBC; "limitations" and help you with the &SQL; problem. It's a sort of middleware to access different data sources. It offers a high level view of data sources and has some places where you can plug in low level access to the database for special tasks.

It offers a wrapper around the database internals, thus making it easier for programmers to make use of all the power provided by many RDBMS without knowing much about it. It comes as a set of libraries, a core one and some extensions (the GTK+ UI extension for example) and some drivers for the most common OpenSource or proprietary database engines.

Some of these drivers are a bit special: the LDAP provider allows one to adress an LDAP directory as if it were composed of tables (provided some configuration), and the &JDBC; provider "wraps" any &JDBC; driver.

Along with these libraries (and associated header files and language bindings for development), &LIBGDA; includes several tools and utilities to help you with the task of developing applications based on &LIBGDA;, as well as for automating some database-related tasks.

&LIBGDA; is implemented for UNIX-like operating systems (including Linux), and Windows.

### Features

This section presents the main features of the 5.2 version of Libgda.

Libgda is a low-level database abstraction layer built on top of each database C API. In terms of abstraction, Libgda offers the following top features:

- Open a connection to any database which type is supported using either a defined data source (defined per user or system wide) or directly providing connection parameters. Several connections can be opened at the same time.

- Run any SQL query against an opened connection; furthermore any SQL query can be extended to define query parameters using an easy to understand notation. If the SQL query was a SELECT statement, then an array of data is returned (as a `GdaDataModel` object), and otherwise the number of rows affected by the query is returned. For very large data sets, it is possible to specify that the returned array of data should be based on a cursor to avoid loading it into memory .

- Perform most of the data definition queries (including database creation) for the database types which support it using a key/value mechanism.

- Create, maintain and extend a view of the database's structure and meta data including the definitions of the tables, constraints, data types, .... It is defined as closely as possible to the information schema SQL standard (ISO/IEC 9075); see the Dictionary - metadata section for more information.

- Easy to extend data models for custom requirements

- Support for binary data and for BLOBs in a transparent way, see the Gda value section

- Support for detailed parameters in SQL queries, see the GdaSqlParser object

- Support for virtual connections, see the Virtual connections chapter

- Partially thread-safe, see the threads limitation for more details

The database types which can be accessed at the time of writing are the following ones:

- Access (.MDB files): uses the MDB Tools library to access the .MDB files, and is limited to getting information about tables and running very simple SELECT queries;

- Firebird: compiles, needs testing;

- MySQL: fully functional;

- Oracle: work is progress, usable;

- PostgreSQL: fully functional;

- SQLite: fully functional;

- Oracle Berkeley DB: fully functional;

- Oracle Berkeley DB with SQL: fully functional;

- SQLCipher: fully functional;

### Architecture

#### Architecture

A &LIBGDA; application is composed of three layers. The lower level layer are the &GDA; providers: plug-ins whose task is to map the RDBMS-specific &API; to the &GDA; model.

Then, in a middle layer, are the libraries provided by &LIBGDA; and used by the programmer: an easy-to-use and full featured set of libraries.

Finally, at the upper layer is the application part itself, as writen by the developer.

Typical &LIBGDA; application's architecture

Architecture of an application connected to 4 databases of 3 different types.

&LIBGDA; also includes:

- some example code which can be "cloned" to create new database providers for database engines not yet supported by &LIBGDA;

- a graphical configuration manager

- an SQL console to connect to dabatase serveurs and perform some &LIBGDA; related administrative tasks

- some other more specific tools

#### Data sources

To open a connection, the programmer needs to specify the database provider to use, and some connection parameters, some of them specific to the database engine to be used, some of them more generic (such as the "DB_NAME" parameter to specify a database name).

All the parameters are combined together to give &LIBGDA; enough information to open a connection; refer to gda_connection_new_from_string() for more information.

However, remembering the complete string to open a connection can be tedious, and so &LIBGDA; supports named data sources (DSN) whereby a connection is specified and named once and opened refering to its name rather than to the actual parameters used. Refer to the Define a data source (DSN) section, and the gda_config_define_dsn() and gda_connection_new_from_dsn() functions.

DSN can have a scope limited to the user, or be system wide. User scope DSN definitions are stored in $XDG_DIR/libgda where $XDG_DIR is determined by the XDG Base Directory Specification (using g_get_user_data_dir()). System wide definitions are stored in $ETC/$VERSION where $ETC is the configuration directory where &LIBGDA; is installed and $VERSION is &LIBGDA;'s major version.

Note that these locations can be changes osing some properties of the global GdaConfig object.

#### Authentication information management

When named data sources are used, &LIBGDA; can also store authentication information (username and password). This is accomplished using libsecret, or libgnome-keyring if available, or stored in an insecure way (along the DSN definition) if none of these libraries are available.

## API reference

The following sections describe the API available for &GDA; applications.

### Object Hierarchy

### Initialization and configuration

Configuration examples are given in this introduction section

#### Configuring &LIBGDA; with environment variables

Some functional aspects of &LIBGDA; can be configured using environment variables which are listed here:

- GDA_CONNECTION_EVENTS_SHOW: if set, then lists the type of events which must be displayed on STDOUT (event types are NOTICE, COMMAND, WARNING, ERROR, the variable can contain a list of types separated by colons, commas or spaces)

- GDA_PROVIDERS_ROOT_DIR: if set, must point to the directory where the providers files are (providers files are shared libraries)

- LIBGDA_NO_THREADS (useless since version 5.2.0): if set, then multi threading will be disabled (see the section about multi threading for more information about &LIBGDA; threads' support). Please note that this environment variables is not used anymore since version 5.2.0, as threading is always enabled by Glib.

- GDA_SHOW_PROVIDER_LOADING_ERROR: if set, then in case a provider fails to be loaded (usually because it requires a library which can't be found) &LIBGDA; will display a warning message. This variable is useful to debug the absence of a provider at runtime

- GDA_DATA_MODEL_DUMP_TITLE: if set, then gda_data_model_dump() will also output the name of the data model as its first line

- GDA_DATA_MODEL_DUMP_ROW_NUMBERS: if set, then gda_data_model_dump() will prepend a column containing row numbers

- GDA_DATA_MODEL_NULL_AS_EMPTY: if set, then gda_data_model_dump() will show an empty ("") string for NULL values, instead of the "NULL" string

- GDA_DATA_MODEL_DUMP_TRUNCATE: if set to a numeric value, then gda_data_model_dump() will truncates the output to the width specified by the value. If the value is -1 then the actual terminal size (if it can be determined) is used

- GDA_DATA_MODEL_DUMP_ATTRIBUTES: if set, then gda_data_model_dump() will also include a description of the data model's attributes

- GDA_CONFIG_SYNCHRONOUS: if set, and if the Gnome Keyring is used to store DSN authentification information, then the requests to the Gnome Keyring will be made synchronously. This is useful if Libgda is used in a program where there is no main loop (otherwise no authentification will be fetched).

- GDA_TREE_DUMP_ALL_ATTRIBUTES: if set, then gda_tree_dump() will dump all the attributes of each node in the tree

- GDAUI_SHOW_PLUGINS_LOADING: if set, then a trace of Gda UI's plugins loading will be output the first time they are loaded.

### Connections & commands

The &LIBGDA; library allows several connections to databases to be opened at once. Once a connection (manipulated through the GdaConnection object) has been created and is opened, it is able to proceed GdaStatement statements.

Each GdaStatement can only hold one SQL statement (such as SELECT, INSERT, CREATE TABLE, BEGIN, etc); statements can be grouped together into GdaBatch batches objects. There are two ways for creating a GdaStatement object:

- either by passing an SQL string to a parser (a GdaSqlParser).

  &LIBGDA; unifies the way variables (also known as "placeholders") are defined (because each database type has its own syntax for doing so); consult the GdaSqlParser object's documentation for a description of this syntax.

- by building it from its structure: see the GdaSqlStatement for reference. However keep in mind that this API is meant to be used primarily by database provider's implementations and for &LIBGDA;'s internal usage. A more user friendly API will be added in the future.

The following schema illustrates how a statement is created and executed: Statement execution illustrated

As each database implements its own SQL variant (all variants more or less close to the SQL92 or SQL99 standards), the &LIBGDA; library allows one to use either a generic SQL parser, or a parser provided by each database adapter (database provider), through the gda_connection_create_parser() method.

The following schema illustrates how a statement is executed: Statement execution illustrated

### Virtual connections

### Data handling

&LIBGDA; being a data oriented library, data handling is a central point of the library:

- individual values are encapsulated within a GValue container (from GLIB)

- value holders which are each a specification for a value: a type, a name, a description, a default value, etc. Value holders are encapsulated within a GdaHolder container object.

  Note that it is possible to set a hint to make a value holder have its value constrained by the values contained in a column of a GdaDataModel.

- lists of value holders which are encapsulated within a GdaSet container object.

  The GdaSet object also makes some computations to group value holders which are constrained by values in the same GdaDataModel (it makes it easy to use value holders which are not really independent).

- arrays of values organized in rows and columns. All the data in the same column have the same type, and all the data in each row have the same semantic meaning. &LIBGDA; uses the GdaDataModel objects to actually hold the data (note that this is actually an interface which has several implementations depending on the real data organization in the data model).

  Note that depending on the real implementation, access to the data can be random or done using an iterator, and that the data model can be read-only or modifiable.

- The GdaDataModelIter object is used to iterate through the rows of a GdaDataModel.

### Trees

&LIBGDA; supports data organized hierarchically in a tree. The specific aspect of this object is that one does not actually create nodes in it directly, but rather specifies the tree structure (ie. what top level nodes represent, whan their children represent, ...) using GdaTreeManager objects: any node in the tree is managed by one and only one such object. This approach makes it very easy to produce trees organized around repetitive data coming from a database. So again, one does not create or remove nodes in a tree, but asks that the tree be specifically updated, in part or starting from a specific node (note that nodes can also be added and removed on events depending on each GdaTreeManager object).

A tree is represented by a unique GdaTree object, in which each node is represented by a GdaTreeNode object. As mentioned previously, any GdaTreeNode in a GdaTree is created (and destroyed) by a GdaTreeManager object: the nodes hierarchy itself is represented by a hierarchy of GdaTreeManager objects (with the one(s) managing the nodes immediately under the tree root attached to the GdaTree itself and the others attached to the GdaTreeManager which act as their parent).

When populating a tree (or updating a part of it), a GdaTreeManager is requested to create (or update) nodes below a parent node. For any node it creates (or updates), and for any sub tree managers it has, it requests those sub managers to populate (or update) the sub nodes of that node; this is a recursive process. Note that the tree manager itself does not store any information about the nodes it has created (this information is stored in the GdaTreeNode themselves): it is stateless (much in the same way as the GtkCellRenderer objects are stateless in a GtkTreeView widget).

The following figure illustrates a simple tree: Overview of the GdaTree The tree has 3 top level nodes (labelled "Node 0", "Node 1" and "Node 2" in the schema), which have been created by the GdaTreeManager labelled "GdaTreeManager 0". These top level nodes have sub nodes (two for the "Node 0" and "Node 1", and none for "Node 2") which have been created by the GdaTreeManager labelled "GdaTreeManager 1"

The following figure illustrates a mode complicated tree which, when dumped is (truncated here for readability):

    .
    |-- information_schema
    |   |-- sql_features
    |   |   |-- feature_name
    |   |   `-- feature_id
    |   `-- sql_parts
    |       |-- comments
    |       `-- feature_id
    |-- public
    |   |-- customers
    |   |   |-- name
    |   |   `-- id
    |   `-- products
    |       |-- name
    |       `-- ref
    |-- customers
    |   |-- name
    |   `-- id
    `-- products
        |-- name
        `-- ref

Overview of the GdaTree The tree has 4 top level nodes ("information_schema", "public" which in fact represent schemas in a PostgreSQL database and "customers" and "products" which are the tables in the "public" schema). Two GdaTreeManager objects are associated to the GdaTree object: the GdaTreeMgrSchemas manager (which creates nodes for each schema in a database) and the GdaTreeMgrTables manager (which creates nodes for each table in a schema, or, if no schema is specified creates nodes for each table visible by default). Here the GdaTreeMgrSchemas manager created the "information_schema" and "public" top level nodes and the GdaTreeMgrTables manager created the "customers" and "products" top level nodes.

The topmost GdaTreeMgrSchemas tree manager object has a child tree manager which is also a GdaTreeManagerTables (and which here is the same object as the GdaTreeManagerTables object which created two of the top level nodes to avoid the unnecessary complication of creating too many tree manager objects). This GdaTreeManagerTables object created the "sql_features", "sql_parts" nodes below the "information_schema" node and the "customers" and "products" nodes below the "public" node.

As the GdaTreeManagerTables object also has a child tree manager, the GdaTreeMgrColumns manager, some nodes are created below the "customers" nodes and below the "products" nodes (some of these nodes are not represented in the figure above for clarity).

As illustrated on the figure above, tree nodes can be assigned named attributes as GValue values. For instance the GdaTreeMgrSchemas manager sets the "schema" attribute to the nodes it creates, the GdaTreeMgrTables manager sets the "table_name" attribute (and the "schema" attribute if the node has been created as a table which is visible by default) and the GdaTreeMgrColumns manager sets the "column_name" attribute to the nodes it creates.

Node attributes can be read and changed at will by the programmer, and are used extensively by the tree manager objects themselves to determine the nodes they nees to create as children of a given node. For example the GdaTreeMgrTables manager, uses gda_tree_node_fetch_attribute() to get the "schema" attribute from the node below which it has to create children nodes. If the attribute exists, it then creates a node for each table in the mentioned schema, and otherwise creates a node for each table visible by default.

The code to set up this tree would be the following one:

    GdaConnection *cnc = ...
    GdaTree *tree;
    GdaTreeManager *schemas_mgr, *tables_mgr, *columns_mgr;

    tree = gda_tree_new ();

    schemas_mgr = gda_tree_mgr_schemas_new (cnc);
    tables_mgr = gda_tree_mgr_tables_new (cnc, NULL);
    columns_mgr = gda_tree_mgr_columns_new (cnc, NULL, NULL);

    gda_tree_manager_add_manager (schemas_mgr, tables_mgr);
    gda_tree_manager_add_manager (tables_mgr, columns_mgr);

    gda_tree_add_manager (tree, schemas_mgr);
    gda_tree_add_manager (tree, tables_mgr);

    g_object_unref (schemas_mgr);
    g_object_unref (tables_mgr);
    g_object_unref (columns_mgr);

    if (!gda_tree_update_all (tree, NULL)) {
            /* ERROR */
    }
    else {
            gda_tree_dump (tree, NULL, NULL);
    }
    g_object_unref (tree);

### Data conversions

A single piece of data can have several representations depending on its usage: a string representation, an SQL representation and of course a GValue representation. Conversions from one representation to the other is DBMS dependent as each database can have its own SQL representation rules. The GdaDataHandler object's purpose is to do all these conversions in a easy way. Except when mentioned otherwise, conversions take into account locale settings and DBMS specifications.

To convert a data, one needs to instantiate a new data handler from one of the many classes which implement this interface, or better to get a pointer to a GdaDataHandler object (no need to unref() it after usage, data handler objects are stateless), and so to obtain such a pointer one can:

- Use the gda_data_handler_get_default(): the returned data handler is a generic one and should not be used to convert data to use with any connection, but only to have a portable way of storing and loading data in a locale independent fashion (for serialization purposes).

- Ask a GdaServerProvider object for one using the gda_server_provider_get_data_handler_g_type() or gda_server_provider_get_data_handler_dbms(): the returned data handler can then safely be used to convert any data to be used with any connection relying on the GdaServerProvider object.

### Dictionary - metadata

Each connection has a dictionary object (a GdaMetaStore) attached to it. That dictionary is either created by the connection when it needs it, or is created and set by the user (to be able to re-use a dictionary), using the "meta-store" property of the GdaConnection object.

Previous versions of &LIBGDA; used an XML file based dictionary which had several drawbacks (see the migrations notes for more details), now a database is used: the default is to use an SQLite base database which is easily transportable and can be used in the same way the previous XML based dictionary was used.

Each GdaMetaStore requires (and creates) a dictionary structure which is a set of tables implementing an "information schema" schema (as defined in the information schema SQL standard (ISO/IEC 9075), and adapted). The user is then free to add more database objects (tables and views) to contain his own data in the dictionary, using the gda_meta_store_schema_add_custom_object() method.

Extracting information can be done using SQL statements on the dictionary database (a special gda_meta_store_extract() method), or, for information about the database structure, using the GdaMetaStruct object which creates an easy to use in-memory representation of some database objects.

#### Setup

Each GdaMetaStore object internally uses a (private) GdaConnection connection object. The following figure illustrates the situation when the programmer uses a GdaConnection connection object when the meta data is stored in a different database (usually an SQLite file):

GdaConnection object and its associated GdaMetaStore using its own database to store the meta data.

From a programmer's point of view, the following code example shows how to get a connection's associated meta store object, without being able to specify anything about the meta store's private connection (in which case the private connection will be an in-memory database destroyed when the meta store object is destroyed):

    GdaConnection *cnc;
    GdaMetaStore *store;

    cnc = gda_connection_open_from_dsn (...);
    g_object_get (G_OBJECT (cnc), "meta-store", &store, NULL);

    [... use the meta store object ...]
    g_object_unref (store);

One can also specify the meta store object to be used by a connection, as in:

    GdaConnection *cnc;
    GdaMetaStore *store;

    cnc = gda_connection_open_from_dsn (...);
    store = gda_meta_store_new_with_file ("/path/to/file");
    /* or */
    store = gda_meta_store_new ("PostgreSQL://DB_NAME=meta_db");

    g_object_set (G_OBJECT (cnc), "meta-store", store, NULL);
    g_object_unref (store);

The meta data can also be stored in the same database as the connection the meta data is for. In this case (and if the associated database provider supports it), the dictionary structure can be placed into a spearate schema. The next figure illustrates this situation:

GdaConnection object and its associated GdaMetaStore using the same database.

From a programmer's point of view, the following code example shows how to do the setup:

    GdaConnection *cnc;
    GdaMetaStore *store;

    cnc = gda_connection_open_from_dsn (...);
    store = GDA_META_STORE (g_object_new (GDA_TYPE_META_STORE, "cnc", cnc, NULL));
    g_object_set (G_OBJECT (cnc), "meta-store", store, NULL);
    g_object_unref (store);

#### Adding custom data

This section explains how to add application specific data (custom data) to a GdaMetaStore object.

Applications of course don't need to use that feature to manage their own data but it makes sense to use it if the application also uses metadata to avoid having to manipulate several files for the same "task".

##### Storing data as (key, value) pairs

In very simple cases, when the data to store is made of named values, the easiest way is to use the gda_meta_store_set_attribute_value() and gda_meta_store_get_attribute_value() methods where the values are stored and retrieved as strings.

##### Declaring the new custom database objects

The gda_meta_store_schema_add_custom_object() method canbe used to add custom objects, which have to be described using an XML syntax. The advantages of using that method are:

- The object's definition will be the same for all database types

- If another object with a different definition already exists, then a error is returned

##### Adding and removing data

For the moment adding and removing data must be done as for any other database connection, using the GdaMetaStore's internal connection obtained with gda_meta_store_get_internal_connection().

##### Removing custom database objects

For the moment adding and removing data must be done as for any other database connection, using the GdaMetaStore's internal connection obtained with gda_meta_store_get_internal_connection().

#### Database structure

&LIBGDA; imposes its own structure (based on objects such as tables and views which name starts with an underscore '_') for the meta data database associated to a GdaMetaStore object, but a user can also add its own tables or other database objects. The following diagram shows the tables defined by &LIBGDA; for its internal needs: Dictionary's tables and views initially defined by &LIBGDA;

##### Data types

Data types (as reported for example for table's columns) can be one of the following kinds:

- Builtin data types offered by the database engine such as integers, varchar, etc. Those data types are all listed in the "_builtin_data_types" table.

- Database domains: domains are data types with optional constraints (restrictions on the allowed set of values). A domain refers to the base data type from which it is defined (that base data type can also be an array). Those data types are all listed in the "_domains" table.

- User defined data types (which are not part of the SQL standard but featured by some databases such as PostgreSQL) are simple or composed data types based on existing data types (which may also be arrays). Those data types are all listed in the "_udt" table.

- Arrays of any of the previous data types (one or more dimensions). Arrays are not per-se data types, but rather refer to an existing data type. Arrays are described in the "_element_types" table.

Note that the data types which are built in, domains or user defined are all also listed in the "_all_types" view as a convenience.

The impact of that data types' hierarchy is that every time a data type is referenced (from a table's column definition for example), there will be two attributes for the data type: one named `data_type` which, if not NULL, refers to a data type listed in the "_all_types" view (thus a built in, domain or user defined data type), and one named `array_spec` which, if not NULL, refers to an array, listed in the "_element_types" table. Any of these attributes can be NULL, but they should never be both NULL at the same time.

Single dimension arrays will have the `data_type` attribute referencing a data type in the "_all_types" view (and the `array_spec` attribute set to NULL), whereas multi dimension arrays will have the `data_type` attribute set to NULL and the `array_spec` attribute referencing another tow in the "_element_types" table.

The following diagram illustrates the data types representation and how they are referred from table's columns, domains, and other database objects which refer to a data type: Dictioanry's tables and views initially defined by &LIBGDA;

##### SQL identifiers

Please refer to the SQL identifiers and abstraction section for an overview of how SQL identifiers are handled thoughout &LIBGDA;

Considering that each database has its own internal way of representing case insensitive SQL identifiers (for instance PostgreSQL represents them in lower case whereas Oracle represents them in upper case and MySQL represents them in upper or lower case depending on the platform and the configuration), &LIBGDA; has chosen by convention to represent case insensitive SQL identifiers in lower case and case sensitive SQL identifiers enclosed between double quotes when storing meta data in GdaMetaStore.

This convention has been made necessary to avoid confusion. To illustrate the problem, let's consider a table named mytable. Without this convention, if that table was in a PostgreSQL database, then the meta data would report it as mytable, and if it was in an ORACLE, it would report it as MYTABLE. Now, a programmer which wants to compare the schemas of the PostgreSQL and Oracle databases has to know that the PostgreSQL's mytable and Oracle's MYTABLE represent in fact the same table which means he needs to know how each database internally stores SQL identifiers, which is not a good choice because it makes the application less portable between database engines. Better let each database provider do the job.

Note that things can even get more confusing if MySQL is used because then one needs to figure out on which platform the MySQL server is running and how it's configured, a task which makes an application even less portable.

To keep the same example, to build some SQL using the SQL identifier as reported by the meta data (mytable for PostgreSQL and MYTABLE for Oracle), one has either to determine if the SQL identifier needs to be quoted or not (which limits the application's portability), or systematically enquote the SQL identifier which is not something &LIBGDA; can impose to application developpers.

##### Short and full names

Database objects will have two associated names, the short name and the long name, the difference being the same as machine in a local network being called either by its machine name such as "moon" or by its fully qualified domain name such as "moon.my.network", but applied to databases and schemas. The full name corresponding to the <database>.<schema>.<object_name> or <schema>.<object_name>, and the short name corresponding to the <object_name>.

Short name and full name differ when the schema being accessed is not the schema containing the named database object, in which case the full name must be used. Some database specific features may also alter the differences between the short name and the sull name such as the "search_path" variable for PostgreSQL (which is to a unix PATH variable but for schemas).

##### Declared foreign keys

Foreign key constraints are used to constraint the contents of one or more columns of a table to the columns of another table. They are also use to help understand the database structure and withing UI programs to present relevant choices.

&LIBGDA; reports existing foreign key constraints in its meta data.

There are however some situations where the database developper does not whish to use foreign keys to actually implement the constraint, but the relation between the tables still exists. &LIBGDA; allows one to declare foreign keys, an operation which does not alter the database's schema, but add information to the database's meta data, specifically into the "__declared_fk" table. For more information, see the gda_meta_store_declare_foreign_key() and gda_meta_store_undeclare_foreign_key methods.

### Data definition (DDL) queries

### Database manipulations

### User interface API reference

### Multi threading

### Miscellaneous

## Using Other Programing Languages

### Introduction

&GDA; has support for &GI;, allowing to use GDA's C API in other languages that support it, like Python or Vala.

### Vala Extensions

Vala is a GObject oriented programing language very similar to C#. &GDA; have added support for through &GI;. Aditionaly some extensions are availabe for better experience using GDA in Vala. You can browse through &GdaVala; documentation.

## &GDA; Tools

### Introduction

&LIBGDA; offers several command line tools to help diagnose and resolve problems. All the provided tools can be run with the "--help" command line argument to give an online help and description. Also, some environment variables can be set to control the output for some of the tools, see &LIBGDA;'s environment variables section for more details.

### &gda-sql;

The &gda-sql; tool is an interactive command line tool to run SQL commands and work with dictionaries. This tool has got several options, use the `--help` option to list them all, or consult the man page (`man gda-sql`)

For example to list the installed providers, do:

    [prompt]> gda-sql -L
                                         Installed providers list
    Provider       | Description
    ---------------+------------------------------------------------------------------------------------
    Berkeley-DB    | Provider for Berkeley databases
    Firebird       | Provider for Firebird databases
    FirebirdEmbed  | Provider for embedded Firebird databases
    Ldap           | Provider for database where tables are based on data contained in an LDAP directory
    MSAccess       | Provider for Microsoft Access files
    [...]

To list the configured data sources (DSN):

    [prompt]> gda-sql -l
                                                        List of defined data sources
    DSN       | Provider | Description                          | Connection string            | Username | Global
    ----------+----------+--------------------------------------+------------------------------+----------+-------
    SalesTest | SQLite   | Test database for a sales department | DB_DIR=/adir;DB_NAME=data.db |          | FALSE
    [...]

To run an interactive session, just specify a DSN or a connection string using the "<provider>://<connection string>" format (such as for example "Firebird://DATABASE=/path/to/dbfile.fdb"), or set the GDA_SQL_CNC environment variable to contain that string, and run the command without any argument, for example:

    [prompt]> gda-sql PostgreSQL://DB_NAME=sales
    Welcome to the GDA SQL console, version 3.99.3

    Type: .copyright to show usage and distribution terms
          .? for help with internal commands
          .q (or CTRL-D) to quit
          (the '.' can be replaced by a '\')
          or any query terminated by a semicolon

    Opening connection 'c0' for: PostgreSQL://DB_NAME=sales
            Getting database schema information, this may take some time... Done.
    c0>

Note that the "c0" is the prompt for the first opened connection.

Several connections can be opened at once (with only one "active" at any given time), by specifying all of them on the command line, or by using the `\c` and `\close` to manage (open, change, close) the connections; the prompt contains the name of the current connection used. The following example opens two connections, one for the "pgsales" DSN, and one for the "PostgreSQL://DB_NAME=sales" connection string, and shows the usage of the ".c" command to list the opened connections:

    [prompt]> gda-sql-6.0 pgsales PostgreSQL://DB_NAME=sales
    Welcome to the GDA SQL console, version 3.99.3

    Type: .copyright to show usage and distribution terms
          .? for help with internal commands
          .q (or CTRL-D) to quit
          (the '.' can be replaced by a '\')
          or any query terminated by a semicolon

    Opening connection 'pgsales' for: pgsales
    Opening connection 'c1' for: PostgreSQL://DB_NAME=sales
            Getting database schema information, this may take some time... Done.
    c1> .c
                 List of opened connections
    Name    | Provider   | DSN or connection string | Username
    --------+------------+--------------------------+---------
    pgsales | PostgreSQL | pgsales                  |
    c1      | PostgreSQL | DB_NAME=sales            |
    (2 rows)

    c1>

Here is another sample session showing how to use variables in statements:

    [prompt]> gda-sql-6.0 -p SQLite -c "DB_DIR=.;DB_NAME=sales_test"
    Welcome to the GDA SQL console, version 3.1.2

    Type: \copyright to show usage and distribution terms
          \? for help with internal commands
          \q (or CTRL-D) to quit
          or any query terminated by a semicolon

    c0> select * from customers;
    id | name            | default_served_by | country | city
    ---+-----------------+-------------------+---------+-----
     2 | Ed Lamton       |                 4 | SP      | MDR
     3 | Lew Bonito      |                 1 | FR      | TLS
     4 | Mark Lawrencep  |              NULL | SP      | MDR
     9 | Greg Popoff     |                 2 | SP      | MDR
    10 | Vladimir Zirkov |                 4 | NULL    | NULL
    c0> \set the_id 9
    c0> select * from customers where id= ##the_id;
    id | name        | default_served_by | country | city
    ---+-------------+-------------------+---------+-----
     9 | Greg Popoff |                 2 | SP      | MDR
    c0>

See the Gda SQL console tool manual section for more information.

### &gda-list-config;

The &gda-list-config; tool simply lists all the declared data sources, and all the installed providers, giving some other useful information as well for each. This tool has no option.

An example output will be:

    [prompt]> gda-list-config-6.0
    === Installed providers ===
    Provider: SQLite
    Description: Provider for SQLite databases
    Location: /usr/lib/libgda-6.0/providers/libgda-sqlite.so
    Data source's parameters (Name / Type / Description):
      DB_NAME / gchararray / Database name
      DB_DIR / gchararray / Directory
      EXTRA_FUNCTIONS / gboolean / Extra functions

    Provider: Berkeley-DB
    Description: Provider for Berkeley databases
    Location: /usr/lib/libgda-6.0/providers/libgda-bdb.so
    Data source's parameters (Name / Type / Description):
      FILE / gchararray / Database file
      DATABASE / gchararray / Database name

    [...]

    === Declared data sources ===
    Data source: Sales
      Key 'DSN' = 'DB_NAME=sales'
      Key 'Description' = 'Sales'
      Key 'Password' = ''
      Key 'Provider' = 'PostgreSQL'
      Key 'Username' = ''

    [...]

### &gda-test-connection;

This tool is a 'ping' like tool to test connectivity to databases. Connections can either be declared as DSN, or described in the command line.

### &gda-list-server-op;

The &gda-list-server-op; tool lists available DDL operations for one or all of the installed providers. For each type of operation (and each provider), the named parameters are also displayed. See the General words about DDL queries for more information about possible operations and usage. This tool has got several options, use the `--help` option to list them all.

For example listing all the possible operations (for all the providers) is:

    [prompt]> gda-list-server-op-6.0 -l
    Using XML descriptions in /usr/share/libgda-6.0
    Existing operation types:
    CREATE_DB
    DROP_DB
    CREATE_TABLE
    DROP_TABLE
    RENAME_TABLE
    ADD_COLUMN
    DROP_COLUMN
    CREATE_INDEX
    DROP_INDEX

Listing all the operations supported by the SQLite provider (notice that the SQLite provider does not support the DROP_COLUMN operation as SQLite does not support it):

    [prompt]> gda-list-server-op-6.0 -l -p SQLite
    Using XML descriptions in /usr/share/libgda-6.0
    For provider SQLite
    Existing operation types for provider 'SQLite':
    CREATE_DB
    DROP_DB
    CREATE_TABLE
    DROP_TABLE
    RENAME_TABLE
    ADD_COLUMN
    CREATE_INDEX
    DROP_INDEX

Listing all the possible parameters for the MySQL provider and for the DROP_COLUMN operation:

    [prompt]> gda-list-server-op-6.0 -o DROP_COLUMN -p MySQL
    Using XML descriptions in /usr/share/libgda-6.0
    For provider MySQL
    Description for type: DROP_COLUMN
    <?xml version="1.0"?>
    <server_op>
      <path id="/COLUMN_DESC_P" node_type="PARAMLIST" name="Column's description"/>
      <path id="/COLUMN_DESC_P/TABLE_NAME" node_type="PARAMETER" gdatype="gchararray" name="Table" descr="Table's name"/>
      <path id="/COLUMN_DESC_P/COLUMN_NAME" node_type="PARAMETER" gdatype="gchararray" name="Field name"/>
    </server_op>

Listing all the possible parameters for the all the installed providers and for the DROP_COLUMN operation:

    [prompt]> gda-list-server-op-6.0 -o DROP_COLUMN
    Using XML descriptions in /usr/share/libgda-6.0
    Description for type: DROP_COLUMN
    <?xml version="1.0"?>
    <server_op>
      <path id="/COLUMN_DESC_P" node_type="PARAMLIST">
        <prov prov_name="mysql" name="Column's description"/>
        <prov prov_name="postgres" name="Column's description"/>
      </path>
      <path id="/COLUMN_DESC_P/TABLE_NAME" node_type="PARAMETER" gdatype="gchararray">
        <prov prov_name="mysql" name="Table" descr="Table's name"/>
        <prov prov_name="postgres" name="Table" descr="Table's name"/>
      </path>
      <path id="/COLUMN_DESC_P/COLUMN_NAME" node_type="PARAMETER" gdatype="gchararray">
        <prov prov_name="mysql" name="Field name"/>
        <prov prov_name="postgres" name="Field name"/>
      </path>
      <path id="/COLUMN_DESC_P/REFERENCED_ACTION" node_type="PARAMETER" gdatype="gchararray">
        <prov prov_name="postgres" name="References" descr="What to do with references on the column to delete"/>
      </path>
    </server_op>

### Control center

This tool is a graphical utility which allows the user to configure and test data sources (define DSN).

### GdaBrowser

This tool is a graphical utility which helps database administrators in their tasks.

## Reports

### Introduction

&LIBGDA;'s report feature has been reworked and is only offers it report engine object, a low level general usage engine to generate reports in the XML format. More specifically it converts an XML tree containing special tags into another XML tree where all the specific tags have been expanded/replaced with database contents. For more information about the special tags taken into account, please refer to the GdaReportEngine's documentation.

Working on any XML file allows the report engine to work with all the existing post-processors which will actually generate the final file(s) in HTML, PDF or other; for example:

- SVG can be processed by a web browser or further edited using inkscape for example

- DocBook can then be converted to HTML of PDF (see this docbook Wiki page)

- RML files (Report Markup Language), see ReportLab or OpenReports can be converted to HTML or PDF.

- some other XML dialects can also be used such as RLib, Papyrus, and other.

To make it easier to use RML or Docbook XML dialects, &LIBGDA; has the GdaReportDocbookDocument and GdaReportRmlDocument which both offer high level of features when dealing with those XML dialects.

From a programmer's point of view, the following code example shows how to convert the "report-spec.xml" file to a "report.xml" report:

    GdaConnection *cnc;
    GdaReportEngine *eng
    xmlDocPtr doc;
    GError *error = NULL;

    cnc = gda_connection_open_from_dsn (...);
    eng = gda_report_engine_new_from_file ("report-spec.xml");
    gda_report_engine_declare_object (eng, G_OBJECT (cnc), "main_cnc");

    doc = gda_report_engine_run_as_doc (eng, &error);
    if (!doc) {
        /* ERROR */
    }
    else {
        xmlSaveFile ("report.xml", doc);
        xmlFreeDoc (doc);
    }
    g_object_unref (eng);

For example the XML "report-spec.xml" file could be:

    <?xml version="1.0" encoding="UTF-8"?>
    <!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.1.2//EN" "http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd">
    <article>
      <articleinfo>
        <title>Customers report example</title>
        <abstract>
          <!-- Use the "abstract" parameter which is supposed to be defined using gda_report_engine_declare_object() -->
          <para><gda_report_param_value param_name="abstract"/></para>
        </abstract>
      </articleinfo>

      <!--
           Start a new section, with the "customers" SELECT query, which is supposed to be defined using
           gda_report_engine_declare_object()

           Any <gda_report_iter> node will repeat its contents as many times as there are rows in the
           data model returned by the "customers" SELECT query.
        -->
      <gda_report_section query_name="customers" cnc_name="main_cnc">
        <sect1>
          <!-- Will be replaced by: <title>List of customers (5)</title> if there are 5 rows for example -->
          <title>List of customers (<gda_report_param_value param_name="customers|?nrows"/>)</title>

          <!-- List all the customer's names -->
          <ul>
    	<gda_report_iter>
    	  <li><gda_report_param_value param_name="customers|@name"/></li>
    	</gda_report_iter>
          </ul>
        </sect1>
      </gda_report_section>
    </article>

For a more detailed example, have a look at the `examples/Report` of &LIBGDA;'s sources.

### API reference

## XSLT extension

### Introduction

&LIBGDA; features an XSLT extension...

### API reference

TODO

### Example

TODO

## Databases providers for developers

For each database engine, &LIBGDA; requires an object which maps &LIBGDA;'s API to the native API offered by that database engine. That object, a database provider needs to inherit GdaServerProvider and implement its virtual methods.

Database provider objects are generally instantiated once by the &LIBGDA; framework and can be used several times to open and work on connections to several databases of the same type.

Since &LIBGDA; itself is developed in the C language, and that most providers are also implemented in that language, the &LIBGDA; library itself contains a set of helper classes and functions to guide you in the addition of a new provider to the GDA framework. Two reference implementations are provided in the `providers/skel-implementation` directory of &LIBGDA;'s sources:

- in `providers/skel-implementation/capi`: a skeleton implementation from scratch for a database accessed through a C API, where all the methods to implement are partially coded

- in `providers/skel-implementation/models`: a skeleton implementation to write a provider for sources which are not databases (which do not implement any SQL) such as the Berkeley DB provider (a BDB database is a key/value collection) or the MDB (MS Access files) provider. This kind of provider "export" tables which are in fact data models.

The helper objects and functions to write database providers are documented in the Providers' support API section.

### Getting started

&LIBGDA;'s sources contain templates to get started in creating a new database provider. The following templates are available:

- the template in the `providers/skel-implementation/capi` directory which can be used when writing a provider using the database's C or C++ API (for example the PostgreSQL or MySQL providers)

- the template in the `providers/skel-implementation/models` directory which can be used when writing a provider for a system which is not a relational database (or does offer a very limited API, such as for the MS Access or Berkeley DB systems).

In any case, for example to create a DummyDb provider, follow these steps:

- copy one of the template's directory into a new directory named `dummydb`

- From inside that new directory, run the `providers/prepare_provider_sources.sh` script with the name of the provider ("dummydb" here), the author's name and the author's email adress, which replaces all the class and object names with the name of the provider and renames the files correctly. These new sources should be compilable without any modification.

- Edit the `Makefile.am` to add provider specific compilation and link flags

- Integrate the provider's new code into a compilation unit: either &LIBGDA;'s sources in the `providers`directory or in your own application (this step usually involves modifying the `configure.ac` or `configure.in` files).

- Implement the missing parts (it is usually a good idea to look how other provider's implementations are done to get ideas).

### Providers' support API

## Packaging

## Appendix

### Index

### Index of new symbols in 4.2

### Index of new symbols in 4.2.1

### Index of new symbols in 4.2.2

### Index of new symbols in 4.2.3

### Index of new symbols in 4.2.4

### Index of new symbols in 4.2.5

### Index of new symbols in 4.2.6

### Index of new symbols in 4.2.7

### Index of new symbols in 4.2.8

### Index of new symbols in 4.2.9

### Index of new symbols in 4.2.10

### Index of new symbols in 5.0

### Index of new symbols in 5.0.2

### Index of new symbols in 5.0.3

### Index of new symbols in 5.2.0

### Index of new symbols in 6.0

### Index of deprecated symbols