database functions

class dballe.DB

DB-All.e database access.

Many methods are the same in dballe.DB and dballe.Transaction. The versions in dballe.DB are implemented by automatically creating a temporary transaction and running the equivalent dballe.Transaction method inside it.

dballe.DB objects are not constructed explicitly, but via one of the DB.connect() or DB.connect_test() class methods.

See Dump the contents of the database and Insert data into a db examples.

attr_insert_data(varid: int, attrs: Dict[str, Any])

Insert new data attributes into the database

attr_insert_station(varid: int, attrs: Dict[str, Any])

Insert new constant station data attributes into the database

attr_query_data(varid: int) → Dict[str, Any]

query data attributes

attr_query_station(varid: int) → Dict[str, Any]

query constant station data attributes

attr_remove_data(varid: int, attrs: Iterable[str])

Remove attributes from data

attr_remove_station(varid: int, attrs: Iterable[str])

Remove attributes from constant station data

connect(url: str) → dballe.DB

create a DB to access a database identified by a DB-All.e URL

connect_from_file(name: str) → dballe.DB

create a DB to access a SQLite file

connect_from_url(url: str) → dballe.DB

create a DB to access a database identified by a DB-All.e URL (deprecated, use connect instead)

connect_test() → dballe.DB

Create a DB for running the test suite, as configured in the test environment

disappear()

Remove all DB-All.e tables and data from the database, if possible

get_default_format() → str

get the default DB format

import_messages(messages: Union[dballe.Message, Sequence[dballe.Message], Iterable[dballe.Message], dballe.ImporterFile], report: str=None, import_attributes: bool=False, update_station: bool=False, overwrite: bool=False)

Import one or more Messages into the database.

Parameters

  • messages –

  • report – the network name to use for importing the data. If left to None, the network is selected automatically from the message type

  • import_attributes – if set to True, requests the variable attributes to also be imported.

  • update_station – if set to True, station information is merged with existing one in the database. If false (default), station information is imported only when the station did not exist in the database.

  • overwrite – if set to True, causes existing information already in the database to be overwritten. If false (default), trying to import a message which contains data already present in the database causes the import to fail.

  • varlist – if set to a string in the same format as the varlist query parameter, only imports data whose varcode is in the list.

insert_data(record: Union[Dict[str, Any], dballe.Cursor, dballe.Data], can_replace: bool=False, can_add_stations: bool=False) → Dict[str, int]

Insert data values in the database

The return value is a dict that always contains ana_id mapped to the station ID just inserted, and an entry for each varcode inserted mapping to the database ID of its value.

insert_station_data(record: Union[Dict[str, Any], dballe.Cursor, dballe.Data], can_replace: bool=False, can_add_stations: bool=False) → Dict[str, int]

Insert station values in the database

The return value is a dict that always contains ana_id mapped to the station ID just inserted, and an entry for each varcode inserted mapping to the database ID of its value.

is_url(url: str) → bool

Checks if a string looks like a DB-All.e DB url

query_data(query: Dict[str, Any]) → dballe.CursorData

Query the data in the database

Returns

a cursor to iterate the query results (see dballe.CursorDataDB)

query_messages(query: Dict[str, Any]) → dballe.CursorMessage

Query the database returning the matching data as Message objects

This can also be used to export messages to a file. For example:

exporter = dballe.Exporter("BUFR")
with open("file.bufr", "wb") as outfile:
    for row in tr.query_messages(...):
        outfile.write(exporter.to_binary(row.message))

See: dballe.Exporter and dballe.CursorMessage.

query_station_data(query: Dict[str, Any]) → dballe.CursorStationData

Query the constant station data in the database

Returns

a cursor to iterate the query results (see dballe.CursorStationDataDB)

query_stations(query: Dict[str, Any]) → dballe.CursorStation

Query the stations in the database

Returns

a cursor to iterate the query results (see dballe.CursorStationDB)

query_summary(query: Dict[str, Any]) → dballe.CursorSummary

Query the summary of the results of a query

Returns

a cursor to iterate the query results (see dballe.CursorSummaryDB)

remove(query: Dict[str, Any])

Remove data variables from the database (deprecated)

remove_all()

Remove all data from the database

remove_data(query: Dict[str, Any])

Remove data variables from the database

remove_station_data(query: Dict[str, Any])

Remove station variables from the database

reset(repinfo_file: str=None)

Reset the database, removing all existing Db-All.e tables and re-creating them empty.

set_default_format(format: str)

set the default DB format

transaction(readonly: bool=False) → dballe.Transaction

Create a new database transaction

vacuum()

Perform database cleanup operations

class dballe.Transaction

DB-All.e transaction

A Transaction is used to execute DB operations in an all-or-nothing fashion. In fact, most DB methods are implemented using a short-lived temporary transaction.

You cannot have more than one active dballe.Transaction for each dballe.DB. An attempt to start a second one will result in an exception being raised. Note that dballe.DB functions like dballe.Transaction.insert_data() create a temporary transaction to run, and so they will also fail if a transaction is currently open. The general idea is that all database work should be done inside a transaction.

Transactions run using the REPEATABLE READ isolation level of the underlying database. This usually means that modifications performed inside a transaction are not visible to other database connections until the transaction is committed. If a transaction is rolled back, all changes done with it are undone.

Transactions can also be used as context managers, which calls dballe.Transaction.commit() automatically, or dballe.Transaction.rollback() if the code raised an exception:

The dballe.Transaction methods are the same as those in dballe.DB. The version in dballe.DB is implemented by automatically creating a temporary transaction and running the dballe.Transaction method inside it.

See Dump the contents of the database and Insert data into a db examples.

attr_insert_data(varid: int, attrs: Dict[str, Any])

Insert new data attributes into the database

attr_insert_station(varid: int, attrs: Dict[str, Any])

Insert new constant station data attributes into the database

attr_query_data(varid: int) → Dict[str, Any]

query data attributes

attr_query_station(varid: int) → Dict[str, Any]

query constant station data attributes

attr_remove_data(varid: int, attrs: Iterable[str])

Remove attributes from data

attr_remove_station(varid: int, attrs: Iterable[str])

Remove attributes from constant station data

commit()

commit the transaction

import_messages(messages: Union[dballe.Message, Sequence[dballe.Message], Iterable[dballe.Message], dballe.ImporterFile], report: str=None, import_attributes: bool=False, update_station: bool=False, overwrite: bool=False)

Import one or more Messages into the database.

Parameters

  • messages –

  • report – the network name to use for importing the data. If left to None, the network is selected automatically from the message type

  • import_attributes – if set to True, requests the variable attributes to also be imported.

  • update_station – if set to True, station information is merged with existing one in the database. If false (default), station information is imported only when the station did not exist in the database.

  • overwrite – if set to True, causes existing information already in the database to be overwritten. If false (default), trying to import a message which contains data already present in the database causes the import to fail.

  • varlist – if set to a string in the same format as the varlist query parameter, only imports data whose varcode is in the list.

insert_data(record: Union[Dict[str, Any], dballe.Cursor, dballe.Data], can_replace: bool=False, can_add_stations: bool=False) → Dict[str, int]

Insert data values in the database

The return value is a dict that always contains ana_id mapped to the station ID just inserted, and an entry for each varcode inserted mapping to the database ID of its value.

insert_station_data(record: Union[Dict[str, Any], dballe.Cursor, dballe.Data], can_replace: bool=False, can_add_stations: bool=False) → Dict[str, int]

Insert station values in the database

The return value is a dict that always contains ana_id mapped to the station ID just inserted, and an entry for each varcode inserted mapping to the database ID of its value.

query_data(query: Dict[str, Any]) → dballe.CursorData

Query the data in the database

Returns

a cursor to iterate the query results (see dballe.CursorDataDB)

query_messages(query: Dict[str, Any]) → dballe.CursorMessage

Query the database returning the matching data as Message objects

This can also be used to export messages to a file. For example:

exporter = dballe.Exporter("BUFR")
with open("file.bufr", "wb") as outfile:
    for row in tr.query_messages(...):
        outfile.write(exporter.to_binary(row.message))

See: dballe.Exporter and dballe.CursorMessage.

query_station_data(query: Dict[str, Any]) → dballe.CursorStationData

Query the constant station data in the database

Returns

a cursor to iterate the query results (see dballe.CursorStationDataDB)

query_stations(query: Dict[str, Any]) → dballe.CursorStation

Query the stations in the database

Returns

a cursor to iterate the query results (see dballe.CursorStationDB)

query_summary(query: Dict[str, Any]) → dballe.CursorSummary

Query the summary of the results of a query

Returns

a cursor to iterate the query results (see dballe.CursorSummaryDB)

remove(query: Dict[str, Any])

Remove data variables from the database (deprecated)

remove_all()

Remove all data from the database

remove_data(query: Dict[str, Any])

Remove data variables from the database

remove_station_data(query: Dict[str, Any])

Remove station variables from the database

rollback()

roll back the transaction