pqxx::stream_from Class Reference

Stream data from the database. More...

#include <stream_from.hxx>

Inheritance diagram for pqxx::stream_from:

Public Types
using	raw_line = std::pair< std::unique_ptr< char, void(*)(void const *)>, std::size_t >

Public Member Functions
	stream_from (stream_from &&)=delete
stream_from &	operator= (stream_from &&)=delete

Static Public Member Functions
static stream_from	query (transaction_base &tx, std::string_view q)
	Factory: Execute query, and stream the results. More...

Streaming data from tables
You can use stream_from to read a table's contents. This is a quick and easy way to read a table, but it comes with limitations. It cannot stream from a view, only from a table. It does not support conditions. And there are no guarantees about ordering. If you need any of those things, consider streaming from a query instead.
	stream_from (transaction_base &, from_query_t, std::string_view query)
	Execute query, and stream over the results. More...
	stream_from (transaction_base &, from_table_t, std::string_view table)
	Stream all rows in table, all columns. More...
template<typename Iter >
	stream_from (transaction_base &, from_table_t, std::string_view table, Iter columns_begin, Iter columns_end)
	Stream given columns from all rows in table. More...
template<typename Columns >
	stream_from (transaction_base &tx, from_table_t, std::string_view table, Columns const &columns)
	Stream given columns from all rows in table. More...
	stream_from (transaction_base &tx, std::string_view table)
template<typename Columns >
	stream_from (transaction_base &tx, std::string_view table, Columns const &columns)
template<typename Iter >
	stream_from (transaction_base &, std::string_view table, Iter columns_begin, Iter columns_end)
	~stream_from () noexcept
constexpr	operator bool () const noexcept
	May this stream still produce more data? More...
constexpr bool	operator! () const noexcept
	Has this stream produced all the data it is going to produce? More...
void	complete ()
	Finish this stream. Call this before continuing to use the connection. More...
template<typename Tuple >
stream_from &	operator>> (Tuple &)
	Read one row into a tuple. More...
template<typename... Vs>
stream_from &	operator>> (std::variant< Vs... > &)=delete
	Doing this with a std::variant is going to be horrifically borked. More...
template<typename... TYPE>
auto	iter () &
	Iterate over this stream. Supports range-based "for" loops. More...
std::vector< zview > const *	read_row () &
	Read a row. Return fields as views, valid until you read the next row. More...
raw_line	get_raw_line ()
	Read a raw line of text from the COPY command. More...
static stream_from	raw_table (transaction_base &tx, std::string_view path, std::string_view columns=""sv)
	Factory: Stream data from a pre-quoted table and columns. More...
static stream_from	table (transaction_base &tx, table_path path, std::initializer_list< std::string_view > columns={})
	Factory: Stream data from a given table. More...

Detailed Description

Stream data from the database.

Deprecated:

Use transaction_base::stream.

For larger data sets, retrieving data this way is likely to be faster than executing a query and then iterating and converting the rows fields. You will also be able to start processing before all of the data has come in.

There are also downsides. Not all kinds of query will work in a stream. But straightforward SELECT and UPDATE ... RETURNING queries should work. This function makes use of pqxx::stream_from, which in turn uses PostgreSQL's COPY command, so see the documentation for those to get the full details.

There are other downsides. If there stream encounters an error, it may leave the entire connection in an unusable state, so you'll have to give the whole thing up. Finally, opening a stream puts the connection in a special state, so you won't be able to do many other things with the connection or the transaction while the stream is open.

There are two ways of starting a stream: you stream either all rows in a table (using one of the factories, table() or raw_table()), or the results of a query (using the query() factory).

Usually you'll want the stream convenience wrapper in transaction_base, * so you don't need to deal with this class directly.

Warning

While a stream is active, you cannot execute queries, open a pipeline, etc. on the same transaction. A transaction can have at most one object of a type derived from pqxx::transaction_focus active on it at a time.

Member Typedef Documentation

raw_line

using pqxx::stream_from::raw_line = std::pair<std::unique_ptr<char, void (*)(void const *)>, std::size_t>

Constructor & Destructor Documentation

stream_from() [1/8]

pqxx::stream_from::stream_from ( stream_from &&  )

delete

stream_from() [2/8]

pqxx::stream_from::stream_from	(	transaction_base &	tx,
		from_query_t	,
		std::string_view	query
	)

Execute query, and stream over the results.

Deprecated:

Use factory function query instead.

References pqxx::transaction_base::exec0(), query(), and pqxx::transaction_focus::register_me().

stream_from() [3/8]

pqxx::stream_from::stream_from	(	transaction_base &	tx,
		from_table_t	,
		std::string_view	table
	)

Stream all rows in table, all columns.

Deprecated:

Use factories table or raw_table instead.

References pqxx::transaction_base::exec0(), pqxx::transaction_base::quote_name(), pqxx::transaction_focus::register_me(), and table().

stream_from() [4/8]

template<typename Iter >

pqxx::stream_from::stream_from	(	transaction_base &	tx,
		from_table_t	,
		std::string_view	table,
		Iter	columns_begin,
		Iter	columns_end
	)

Stream given columns from all rows in table.

Deprecated:

Use factories table or raw_table instead.

stream_from() [5/8]

template<typename Columns >

pqxx::stream_from::stream_from	(	transaction_base &	tx,
		from_table_t	,
		std::string_view	table,
		Columns const &	columns
	)

Stream given columns from all rows in table.

Deprecated:

Use factory function query instead.

stream_from() [6/8]

pqxx::stream_from::stream_from	(	transaction_base &	tx,
		std::string_view	table
	)

Deprecated:

Use factories table or raw_table instead.

stream_from() [7/8]

template<typename Columns >

pqxx::stream_from::stream_from	(	transaction_base &	tx,
		std::string_view	table,
		Columns const &	columns
	)

Deprecated:

Use factories table or raw_table instead.

stream_from() [8/8]

template<typename Iter >

pqxx::stream_from::stream_from	(	transaction_base &	,
		std::string_view	table,
		Iter	columns_begin,
		Iter	columns_end
	)

Deprecated:

Use factories table or raw_table instead.

~stream_from()

pqxx::stream_from::~stream_from ( )

noexcept

Member Function Documentation

complete()

void pqxx::stream_from::complete

(

)

Finish this stream. Call this before continuing to use the connection.

Consumes all remaining lines, and closes the stream.

This may take a while if you're abandoning the stream before it's done, so skip it in error scenarios where you're not planning to use the connection again afterwards.

References pqxx::ignore_unused().

get_raw_line()

pqxx::stream_from::raw_line pqxx::stream_from::get_raw_line

(

)

Read a raw line of text from the COPY command.

Warning

Do not use this unless you really know what you're doing.

Referenced by pqxx::stream_to::operator<<().

iter()

template<typename... TYPE>

auto pqxx::stream_from::iter

(

)

&

Iterate over this stream. Supports range-based "for" loops.

Produces an input iterator over the stream.

Do not call this yourself. Use it like "for (auto data : stream.iter())".

operator bool()

constexpr pqxx::stream_from::operator bool ( ) const

constexprnoexcept

May this stream still produce more data?

operator!()

constexpr bool pqxx::stream_from::operator! ( ) const

constexprnoexcept

Has this stream produced all the data it is going to produce?

operator=()

stream_from& pqxx::stream_from::operator= ( stream_from &&  )

delete

operator>>() [1/2]

template<typename... Vs>

stream_from& pqxx::stream_from::operator>> ( std::variant< Vs... > &  )

delete

Doing this with a std::variant is going to be horrifically borked.

operator>>() [2/2]

template<typename Tuple >

stream_from & pqxx::stream_from::operator>>

(

Tuple &

t

)

Read one row into a tuple.

Converts the row's fields into the fields making up the tuple.

For a column which can contain nulls, be sure to give the corresponding tuple field a type which can be null. For example, to read a field as int when it may contain nulls, read it as std::optional<int>. Using std::shared_ptr or std::unique_ptr will also work.

query()

static stream_from pqxx::stream_from::query ( transaction_base &  tx, std::string_view  q  )

static

Factory: Execute query, and stream the results.

The query can be a SELECT query or a VALUES query; or it can be an UPDATE, INSERT, or DELETE with a RETURNING clause.

The query is executed as part of a COPY statement, so there are additional restrictions on what kind of query you can use here. See the PostgreSQL documentation for the COPY command:

https://www.postgresql.org/docs/current/sql-copy.html

References pqxx::from_query.

Referenced by stream_from().

raw_table()

pqxx::stream_from pqxx::stream_from::raw_table ( transaction_base &  tx, std::string_view  path, std::string_view  columns = ""sv  )

static

Factory: Stream data from a pre-quoted table and columns.

Use this factory if you need to create multiple streams using the same table path and/or columns list, and you want to save a bit of work on composing the internal SQL statement for starting the stream. It lets you compose the string representations for the table path and the columns list, so you can compute these once and then re-use them later.

Parameters

tx	The transaction within which the stream will operate.
path	Name or path for the table upon which the stream will operate. If any part of the table path may contain special characters or be case-sensitive, quote the path using pqxx::connection::quote_table().
columns	Columns which the stream will read. They should be comma-separated and, if needed, quoted. You can produce the string using pqxx::connection::quote_columns(). If you omit this argument, the stream will read all columns in the table, in schema order.

References pqxx::from_table.

read_row()

std::vector< pqxx::zview > const * pqxx::stream_from::read_row

(

)

&

Read a row. Return fields as views, valid until you read the next row.

Returns nullptr when there are no more rows to read. Do not attempt to read any further rows after that.

Do not access the vector, or the storage referenced by the views, after closing or completing the stream, or after attempting to read a next row.

A pqxx::zview is like a std::string_view, but with the added guarantee that if its data pointer is non-null, the string is followed by a terminating zero (which falls just outside the view itself).

If any of the views' data pointer is null, that means that the corresponding SQL field is null.

Warning

The return type may change in the future, to support C++20 coroutine-based usage.

table()

pqxx::stream_from pqxx::stream_from::table ( transaction_base &  tx, table_path  path, std::initializer_list< std::string_view >  columns = {}  )

static

Factory: Stream data from a given table.

This is the convenient way to stream from a table.

References pqxx::transaction_base::conn().

Referenced by stream_from().

---

The documentation for this class was generated from the following files:

• stream_from.hxx
• stream_from.cxx