04_ais_stream_db.c File Reference

A simple program that reads AIS data from a CSV file, accumulates the observations in main memory and send the temporal values to a MobilityDB database when they reach a given number of instants in order to free the memory and ingest the newest observations. More...

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#include <libpq-fe.h>
#include <meos.h>
#include <meos_geo.h>
#include <meos_internal.h>

Data Structures
struct	AIS_record
struct	trip_record

Macros
#define	MAX_LENGTH_HEADER   1024
#define	MAX_LENGTH_POINT   64
#define	MAX_TRIPS   5
#define	NO_INSTANTS_BATCH   1000
#define	NO_INSTANTS_KEEP   2

Functions
static int	exec_sql (PGconn *conn, const char *sql, ExecStatusType status)
	Function that sends a SQL query to the database. More...
int	main (int argc, char **argv)

Detailed Description

A simple program that reads AIS data from a CSV file, accumulates the observations in main memory and send the temporal values to a MobilityDB database when they reach a given number of instants in order to free the memory and ingest the newest observations.

This program is similar to 04_ais_store but illustrates the use of MEOS expandable data structures, which were designed to cope with the requirements of stream applications. In this setting, the expandable data structures accumulate the observations that have been received so far. Depending on application requirements and the available memory, the accumulated temporal values must be sent regularly to the database.

This program uses the libpq library https://www.postgresql.org/docs/current/libpq.html for connecting to a PostgreSQL database that has the MobilityDB extension. For this, it is required that libpq-dev is installed in your system. For example, in Ubuntu you can install this library as follows

sudo apt-get install libpq-dev

You should configure PostgreSQL to accept all incoming connections by setting in the pg_hba.conf file

host all all 0.0.0.0/0 md5

Also, make sure that PostgreSQL allows incoming connections on all available IP interfaces by setting in the postgresql.conf file

listen_addresses = '*'

This program is based on the libpq example programs https://www.postgresql.org/docs/current/libpq-example.html Please read the assumptions made about the input file in the file 02_ais_read.c in the same directory.

The program can be build as follows

gcc -Wall -g -I/usr/local/include -I/usr/include/postgresql -o 04_ais_stream_db 04_ais_stream_db.c -L/usr/local/lib -lmeos -lpq