library db-extractor-common {🖫 2,755, ⚐ 84, ★ 0}

keboola/db-extractor-common

Common library from Keboola Database Extractors

Thursday, June 14, 2018
by Keboola
Repository
5 Watchers
0 Stars
2,755 Installations

PHP
0 Dependents
0 Suggesters
0 Forks
8 Open issues
84 Versions
5 % Grown

The README.md

# Database Extractor Common [DBC]

Common classes for creating vendor specific database extractors., _(*1)

Extractors using DBC

MySQL
MSSQL
PgSQL
Oracle (private repository)
Impala
Firebird
DB2
Redshift
Snowflake

Development and running tests

docker compose build
docker compose run --rm tests  # runs the tests

Usage

Add the library to your component's composer:, _(*2)

php composer.phar require db-extractor-common

composer.json, _(*3)

{
  "require": "db-extractor-common": ^8.0
}

Usage

Create entrypoint script file run.php like this one for Mysql extractor: https://github.com/keboola/db-extractor-mysql/blob/master/src/run.php, _(*4)

Note that as of version 7, configuration rows are supported so it is not necessary to support .yml configs or table array configurations., _(*5)

The $config is loaded from the config.json file. You have to provide values for the data_dir and extractor_class keys. extractor_class is the main class of derived extractor, it should extend Keboola\DbExtractor\Extractor\Extractor., _(*6)

You will need to implement the following methods: - createConnection(array $params) - testConnection() - simpleQuery(array $table, array $columns = array()): string - getTables(?array $tables = null): array;, _(*7)

Note that to support identifier sanitation, the getTables method should return a sanitizedName property for each column. This sanitizedName should be created using Keboola\php-utils\sanitizeColumnName, _(*8)

If you want to implement incremental fetching, you must implement
validateIncrementalFetching(array $table, string $columnName, ?int $limit = null): void
Please see the sample in the Common class: https://github.com/keboola/db-extractor-common/blob/master/src/Keboola/DbExtractor/Extractor/Common.php#L52, _(*9)

The namespace of your extractor class shoud be Keboola\DbExtractor\Extractor and the name of the class should corespond to DB vendor name i.e. PgSQL, Oracle, Impala, Firebrid, DB2 and so on., _(*10)

Please check the existing implementations above for help getting started., _(*11)

Check username

Check username functionality compares - value of the KBC_REALUSER environment variable - and username of the database user from the configuration., _(*12)

If enabled, values must be same to run the component., _(*13)

If a service account is used, check is skipped. For more information see UsernameChecker class., _(*14)

Example image / stack parameters, _(*15)

{
  "checkUsername": {
    "enabled": true
  }
}

Service account can be defined by regular expression:, _(*16)

{
  "checkUsername": {
    "enabled": true,
    "serviceAccountRegexp": "~^service_~i"
  }
}

Or user account can be defined by regular expression (inverted logic):, _(*17)

{
  "checkUsername": {
    "enabled": true,
    "userAccountRegexp": "~^[0-9]{3}_"
  }
}

License

MIT licensed, see LICENSE file., _(*18)

The Versions

14/06 2018

dev-piv-empty-table-fix

Common library from Keboola Database Extractors

library db-extractor-common

Common library from Keboola Database Extractors

keboola/db-extractor-common

The README.md

# Database Extractor Common [DBC]

Extractors using DBC

Development and running tests

Usage

Usage

Check username

License

The Versions

dev-piv-empty-table-fix

The Requires

The Development Requires

by Keboola

dev-master

The Requires

The Development Requires

by Keboola

by Miroslav Čillík

9.1.3

The Requires

The Development Requires

by Keboola

9.1.2

The Requires

The Development Requires

by Keboola

9.1.1

The Requires

The Development Requires

by Keboola

dev-piv-update-logger

The Requires

The Development Requires

by Keboola

9.1.0

The Requires

The Development Requires

by Keboola

dev-piv-retry-ssh

The Requires

The Development Requires

by Keboola

dev-pivnicek-expose-dbParameters

The Requires

The Development Requires

by Keboola

9.0.0

The Requires

The Development Requires

by Keboola

dev-piv-remove-constants

The Requires

The Development Requires

by Keboola

dev-piv-refactor-ssh-keys

The Requires

The Development Requires

by Keboola

dev-piv-retry-refactor

The Requires

The Development Requires

by Keboola

dev-piv-CI-update

The Requires

The Development Requires

by Keboola

dev-piv-empty-row-packet

The Requires

The Development Requires

by Keboola

8.1.0

The Requires

The Development Requires

by Keboola

8.0.0

The Requires

The Development Requires