Fast db insert with postgresql binary copy


Keywords
bulk-loader, postgresql
License
MIT
Install
pip install pgcopy==1.6.0

Documentation

pgcopy

image

image

image

image

image

Use pgcopy for fast data loading into PostgreSQL with binary copy.

Features

  • Support for many data types
  • Support for multi-dimensional array types
  • Support for schema and schema search path
  • Support for mixed-case table and column names
  • Transparent string encoding
  • Utility for replacing entire table

Quickstart

from datetime import datetime
from pgcopy import CopyManager
import psycopg2
cols = ('id', 'timestamp', 'location', 'temperature')
now = datetime.now()
records = [
        (0, now, 'Jerusalem', 72.2),
        (1, now, 'New York', 75.6),
        (2, now, 'Moscow', 54.3),
    ]
conn = psycopg2.connect(database='weather_db')
mgr = CopyManager(conn, 'measurements_table', cols)
mgr.copy(records)

# don't forget to commit!
conn.commit()

Supported datatypes

pgcopy supports the following PostgreSQL scalar types:

  • bool
  • smallint
  • integer
  • bigint
  • real
  • double precision
  • char
  • varchar
  • text
  • bytea
  • enum types
  • date
  • time
  • timestamp
  • timestamp with time zone
  • numeric
  • json
  • jsonb
  • uuid
  • arrays
  • vector

Documentation

Read the docs.

See Also

cpgcopy, a Cython implementation, about twice as fast.