Howdy all,
I'm trying to import data from Postgres, and I seem to be hitting the following error:
Stderr: 'org.neo4j.internal.batchimport.input.HeaderException: Group 'public._product_collection_filters' not found
. Available groups are: [ <redacted for brevity>
This is obviously caused by the table _product_collection_filters. At first I thought it might be a strange case of snake case to camelcase or similar, but the only references to product_collection_filters of any kind are in:
- the
mapping.json, within the sql it generates, under a table key, under an id-space key, and under a name key (as PRODUCT_COLLECTION_FILTERS)
- the headers file (
REL_PRODUCTCOLLECTIONFILTERS_34d0ff28-4caf-4698-9a4d-edf456417552_headers.csv), in the END_ID header - :END_I D(public._product_collection_filters)
- the actual data csv export, in what would be the
:TYPE column, with value PRODUCT_COLLECTION_FILTER
The command I'm running to run the mapping is:
${etl_tool} generate-metadata-mapping \
--rdbms:url ${RDBMS_URL} \
--rdbms:user ${RDBMS_USER} \
--rdbms:password ${RDBMS_PASSWORD} \
--output-mapping-file /data/mapping.json
--exclusion-mode-tables ${TABLES_EXCLUSION_MODE} \
--tables \"-l ${TABLES}\"
and the export command is:
${etl_tool} export \
--rdbms:url ${RDBMS_URL} \
--rdbms:user ${RDBMS_USER} \
--rdbms:password ${RDBMS_PASSWORD} \
--neo4j:user ${NEO4J_USER} \
--neo4j:password ${NEO4J_PASSWORD} \
--using bulk:neo4j-import \
--import-tool ${NEO4J_HOME}/bin \
--csv-directory /data/csv \
--options-file /data/options.json \
--mapping-file /data/mapping.json \
--destination ${NEO4J_HOME}/data/databases/neo4j/ \
--exclusion-mode-tables ${TABLES_EXCLUSION_MODE} \
--tables \"-l ${TABLES}\"
--quote '"' \
--force
I have saved a copy of the CSV data & mapping and the command that the etl tool generates to import the CSVs. I'm not sure where the logs are saved to have those available; I've looked at /var/lib/neo4j/import/import.report and /var/lib/neo4j/logs but there's nothing there.
Additional info:
- Neo4j version v4.0.4 (Docker)
- ETL tool version v1.5.0 (I've packaged this into the same Docker image)
- Importing from Postgres
Also, I wanted to try excluding this table but it looks like the mapping & export commands don't seem to be respecting the table inclusions/exclusions I've provided. Not sure if it's worth creating an issue or new post for this?
