This website uses cookies. By clicking Accept, you consent to the use of cookies. Click Here to learn more about how we use cookies.
Accept
Reject
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help

Head's Up! These forums are read-only. All users and content have migrated. Please join us at community.neo4j.com.

Neo4j and Snowflake Cloud Database

lpfrnk
Node

‎07-03-2019 05:05 AM

Trying to get Neo4j ETL to function with Snowflake. Is the source code available? or can the connection be evaluated? The system is unable to read the tables. It is able to identify the available schemas.

  • 90.9% - 00:00:00.010 -

  • 9.1% - 00:00:00.001 -

  • Crawling tables

  • Retrieving tables

  • Retrieving tables for schema <xxx.LXPRDUSRF>

  • Could not log SQL warnings for result set
    net.snowflake.client.jdbc.SnowflakeSQLException: Result set has been closed.
    at net.snowflake.client.jdbc.SnowflakeBaseResultSet.raiseSQLExceptionIfResultSetIsClosed(SnowflakeBaseResultSet.java:74)
    at net.snowflake.client.jdbc.SnowflakeBaseResultSet.getWarnings(SnowflakeBaseResultSet.java:286)
    at sf.util.DatabaseUtility.logSQLWarnings(DatabaseUtility.java:248)
    at schemacrawler.crawl.MetadataResultSet.next(MetadataResultSet.java:485)
    at schemacrawler.crawl.TableRetriever.retrieveTablesFromMetadata(TableRetriever.java:242)
    at schemacrawler.crawl.TableRetriever.retrieveTables(TableRetriever.java:118)
    at schemacrawler.crawl.SchemaCrawler.lambda$crawlTables$17(SchemaCrawler.java:585)
    at sf.util.StopWatch.time(StopWatch.java:185)
    at schemacrawler.crawl.SchemaCrawler.crawlTables(SchemaCrawler.java:582)
    at schemacrawler.crawl.SchemaCrawler.crawl(SchemaCrawler.java:817)
    at schemacrawler.utility.SchemaCrawlerUtility.getCatalog(SchemaCrawlerUtility.java:75)
    at org.neo4j.etl.sql.DatabaseClient.getCatalog(DatabaseClient.java:148)
    at org.neo4j.etl.sql.DatabaseCatalog.(DatabaseCatalog.java:81)
    at org.neo4j.etl.sql.metadata.TableInfoAssembler.(TableInfoAssembler.java:35)
    at org.neo4j.etl.commands.DatabaseInspector.buildSchemaExport(DatabaseInspector.java:35)
    at org.neo4j.etl.commands.rdbms.GenerateMetadataMapping.call(GenerateMetadataMapping.java:110)
    at org.neo4j.etl.commands.rdbms.GenerateMetadataMapping.call(GenerateMetadataMapping.java:31)
    at org.neo4j.etl.cli.rdbms.GenerateMetadataMappingCli.run(GenerateMetadataMappingCli.java:118)
    at org.neo4j.etl.util.CliRunner.run(CliRunner.java:42)
    at org.neo4j.etl.util.CliRunner.run(CliRunner.java:35)
    at org.neo4j.etl.NeoIntegrationCli.main(NeoIntegrationCli.java:43)

  • Processed 33 rows for

  • Retrieved 33 tables

  • Retrieving table columns

  • Could not log SQL warnings for result set
    net.snowflake.client.jdbc.SnowflakeSQLException: Result set has been closed.
    at net.snowflake.client.jdbc.SnowflakeBaseResultSet.raiseSQLExceptionIfResultSetIsClosed(SnowflakeBaseResultSet.java:74)
    at net.snowflake.client.jdbc.SnowflakeBaseResultSet.getWarnings(SnowflakeBaseResultSet.java:286)
    at sf.util.DatabaseUtility.logSQLWarnings(DatabaseUtility.java:248)
    at schemacrawler.crawl.MetadataResultSet.next(MetadataResultSet.java:485)
    at schemacrawler.crawl.TableColumnRetriever.retrieveColumnsFromMetadata(TableColumnRetriever.java:304)
    at schemacrawler.crawl.TableColumnRetriever.retrieveColumns(TableColumnRetriever.java:107)
    at schemacrawler.crawl.SchemaCrawler.lambda$crawlTables$18(SchemaCrawler.java:603)
    at sf.util.StopWatch.time(StopWatch.java:185)
    at schemacrawler.crawl.SchemaCrawler.crawlTables(SchemaCrawler.java:600)
    at schemacrawler.crawl.SchemaCrawler.crawl(SchemaCrawler.java:817)
    at schemacrawler.utility.SchemaCrawlerUtility.getCatalog(SchemaCrawlerUtility.java:75)
    at org.neo4j.etl.sql.DatabaseClient.getCatalog(DatabaseClient.java:148)
    at org.neo4j.etl.sql.DatabaseCatalog.(DatabaseCatalog.java:81)
    at org.neo4j.etl.sql.metadata.TableInfoAssembler.(TableInfoAssembler.java:35)
    at org.neo4j.etl.commands.DatabaseInspector.buildSchemaExport(DatabaseInspector.java:35)
    at org.neo4j.etl.commands.rdbms.GenerateMetadataMapping.call(GenerateMetadataMapping.java:110)
    at org.neo4j.etl.commands.rdbms.GenerateMetadataMapping.call(GenerateMetadataMapping.java:31)
    at org.neo4j.etl.cli.rdbms.GenerateMetadataMappingCli.run(GenerateMetadataMappingCli.java:118)
    at org.neo4j.etl.util.CliRunner.run(CliRunner.java:42)
    at org.neo4j.etl.util.CliRunner.run(CliRunner.java:35)
    at org.neo4j.etl.NeoIntegrationCli.main(NeoIntegrationCli.java:43)

  • Could not log SQL warnings for result set
    net.snowflake.client.jdbc.SnowflakeSQLException: Result set has been closed.
    at net.snowflake.client.jdbc.SnowflakeBaseResultSet.raiseSQLExceptionIfResultSetIsClosed(SnowflakeBaseResultSet.java:74)
    at net.snowflake.client.jdbc.SnowflakeBaseResultSet.getWarnings(SnowflakeBaseResultSet.java:286)
    at sf.util.DatabaseUtility.logSQLWarnings(DatabaseUtility.java:248)
    at schemacrawler.crawl.MetadataResultSet.next(MetadataResultSet.java:485)
    at schemacrawler.crawl.TableColumnRetriever.retrieveColumnsFromMetadata(TableColumnRetriever.java:304)
    at schemacrawler.crawl.TableColumnRetriever.retrieveColumns(TableColumnRetriever.java:107)
    at schemacrawler.crawl.SchemaCrawler.lambda$crawlTables$18(SchemaCrawler.java:603)
    at sf.util.StopWatch.time(StopWatch.java:185)
    at schemacrawler.crawl.SchemaCrawler.crawlTables(SchemaCrawler.java:600)
    at schemacrawler.crawl.SchemaCrawler.crawl(SchemaCrawler.java:817)
    at schemacrawler.utility.SchemaCrawlerUtility.getCatalog(SchemaCrawlerUtility.java:75)
    at org.neo4j.etl.sql.DatabaseClient.getCatalog(DatabaseClient.java:148)
    at org.neo4j.etl.sql.DatabaseCatalog.(DatabaseCatalog.java:81)
    at org.neo4j.etl.sql.metadata.TableInfoAssembler.(TableInfoAssembler.java:35)
    at org.neo4j.etl.commands.DatabaseInspector.buildSchemaExport(DatabaseInspector.java:35)
    at org.neo4j.etl.commands.rdbms.GenerateMetadataMapping.call(GenerateMetadataMapping.java:110)
    at org.neo4j.etl.commands.rdbms.GenerateMetadataMapping.call(GenerateMetadataMapping.java:31)
    at org.neo4j.etl.cli.rdbms.GenerateMetadataMappingCli.run(GenerateMetadataMappingCli.java:118)
    at org.neo4j.etl.util.CliRunner.run(CliRunner.java:42)
    at org.neo4j.etl.util.CliRunner.run(CliRunner.java:35)
    at org.neo4j.etl.NeoIntegrationCli.main(NeoIntegrationCli.java:43)

  • Could not log SQL warnings for result set
    net.snowflake.client.jdbc.SnowflakeSQLException: Result set has been closed.
    at net.snowflake.client.jdbc.SnowflakeBaseResultSet.raiseSQLExceptionIfResultSetIsClosed(SnowflakeBaseResultSet.java:74)
    at net.snowflake.client.jdbc.SnowflakeBaseResultSet.getWarnings(SnowflakeBaseResultSet.java:286)
    at sf.util.DatabaseUtility.logSQLWarnings(DatabaseUtility.java:248)
    at schemacrawler.crawl.MetadataResultSet.next(MetadataResultSet.java:485)
    at schemacrawler.crawl.TableColumnRetriever.retrieveColumnsFromMetadata(TableColumnRetriever.java:304)
    at schemacrawler.crawl.TableColumnRetriever.retrieveColumns(TableColumnRetriever.java:107)
    at schemacrawler.crawl.SchemaCrawler.lambda$crawlTables$18(SchemaCrawler.java:603)
    at sf.util.StopWatch.time(StopWatch.java:185)
    at schemacrawler.crawl.SchemaCrawler.crawlTables(SchemaCrawler.java:600)
    at schemacrawler.crawl.SchemaCrawler.crawl(SchemaCrawler.java:817)
    at schemacrawler.utility.SchemaCrawlerUtility.getCatalog(SchemaCrawlerUtility.java:75)
    at org.neo4j.etl.sql.DatabaseClient.getCatalog(DatabaseClient.java:148)
    at org.neo4j.etl.sql.DatabaseCatalog.(DatabaseCatalog.java:81)
    at org.neo4j.etl.sql.metadata.TableInfoAssembler.(TableInfoAssembler.java:35)
    at org.neo4j.etl.commands.DatabaseInspector.buildSchemaExport(DatabaseInspector.java:35)
    at org.neo4j.etl.commands.rdbms.GenerateMetadataMapping.call(GenerateMetadataMapping.java:110)
    at org.neo4j.etl.commands.rdbms.GenerateMetadataMapping.call(GenerateMetadataMapping.java:31)
    at org.neo4j.etl.cli.rdbms.GenerateMetadataMappingCli.run(GenerateMetadataMappingCli.java:118)
    at org.neo4j.etl.util.CliRunner.run(CliRunner.java:42)
    at org.neo4j.etl.util.CliRunner.run(CliRunner.java:35)
    at org.neo4j.etl.NeoIntegrationCli.main(NeoIntegrationCli.java:43)

  • Could not log SQL warnings for result set
    net.snowflake.client.jdbc.SnowflakeSQLException: Result set has been closed.
    at net.snowflake.client.jdbc.SnowflakeBaseResultSet.raiseSQLExceptionIfResultSetIsClosed(SnowflakeBaseResultSet.java:74)
    at net.snowflake.client.jdbc.SnowflakeBaseResultSet.getWarnings(SnowflakeBaseResultSet.java:286)
    at sf.util.DatabaseUtility.logSQLWarnings(DatabaseUtility.java:248)
    at schemacrawler.crawl.MetadataResultSet.next(MetadataResultSet.java:485)
    at schemacrawler.crawl.TableColumnRetriever.retrieveColumnsFromMetadata(TableColumnRetriever.java:304)
    at schemacrawler.crawl.TableColumnRetriever.retrieveColumns(TableColumnRetriever.java:107)
    at schemacrawler.crawl.SchemaCrawler.lambda$crawlTables$18(SchemaCrawler.java:603)
    at sf.util.StopWatch.time(StopWatch.java:185)
    at schemacrawler.crawl.SchemaCrawler.crawlTables(SchemaCrawler.java:600)
    at schemacrawler.crawl.SchemaCrawler.crawl(SchemaCrawler.java:817)
    at schemacrawler.utility.SchemaCrawlerUtility.getCatalog(SchemaCrawlerUtility.java:75)
    at org.neo4j.etl.sql.DatabaseClient.getCatalog(DatabaseClient.java:148)
    at org.neo4j.etl.sql.DatabaseCatalog.(DatabaseCatalog.java:81)
    at org.neo4j.etl.sql.metadata.TableInfoAssembler.(TableInfoAssembler.java:35)
    at org.neo4j.etl.commands.DatabaseInspector.buildSchemaExport(DatabaseInspector.java:35)
    at org.neo4j.etl.commands.rdbms.GenerateMetadataMapping.call(GenerateMetadataMapping.java:110)
    at org.neo4j.etl.commands.rdbms.GenerateMetadataMapping.call(GenerateMetadataMapping.java:31)
    at org.neo4j.etl.cli.rdbms.GenerateMetadataMappingCli.run(GenerateMetadataMappingCli.java:118)
    at org.neo4j.etl.util.CliRunner.run(CliRunner.java:42)
    at org.neo4j.etl.util.CliRunner.run(CliRunner.java:35)
    at org.neo4j.etl.NeoIntegrationCli.main(NeoIntegrationCli.java:43)

  • Could not log SQL warnings for result set
    net.snowflake.client.jdbc.SnowflakeSQLException: Result set has been closed.
    at net.snowflake.client.jdbc.SnowflakeBaseResultSet.raiseSQLExceptionIfResultSetIsClosed(SnowflakeBaseResultSet.java:74)
    at net.snowflake.client.jdbc.SnowflakeBaseResultSet.getWarnings(SnowflakeBaseResultSet.java:286)
    at sf.util.DatabaseUtility.logSQLWarnings(DatabaseUtility.java:248)
    at schemacrawler.crawl.MetadataResultSet.next(MetadataResultSet.java:485)
    at schemacrawler.crawl.TableColumnRetriever.retrieveColumnsFromMetadata(TableColumnRetriever.java:304)
    at schemacrawler.crawl.TableColumnRetriever.retrieveColumns(TableColumnRetriever.java:107)
    at schemacrawler.crawl.SchemaCrawler.lambda$crawlTables$18(SchemaCrawler.java:603)
    at sf.util.StopWatch.time(StopWatch.java:185)
    at schemacrawler.crawl.SchemaCrawler.crawlTables(SchemaCrawler.java:600)
    at schemacrawler.crawl.SchemaCrawler.crawl(SchemaCrawler.java:817)
    at schemacrawler.utility.SchemaCrawlerUtility.getCatalog(SchemaCrawlerUtility.java:75)
    at org.neo4j.etl.sql.DatabaseClient.getCatalog(DatabaseClient.java:148)
    at org.neo4j.etl.sql.DatabaseCatalog.(DatabaseCatalog.java:81)
    at org.neo4j.etl.sql.metadata.TableInfoAssembler.(TableInfoAssembler.java:35)
    at org.neo4j.etl.commands.DatabaseInspector.buildSchemaExport(DatabaseInspector.java:35)
    at org.neo4j.etl.commands.rdbms.GenerateMetadataMapping.call(GenerateMetadataMapping.java:110)
    at org.neo4j.etl.commands.rdbms.GenerateMetadataMapping.call(GenerateMetadataMapping.java:31)
    at org.neo4j.etl.cli.rdbms.GenerateMetadataMappingCli.run(GenerateMetadataMappingCli.java:118)
    at org.neo4j.etl.util.CliRunner.run(CliRunner.java:42)
    at org.neo4j.etl.util.CliRunner.run(CliRunner.java:35)
    at org.neo4j.etl.NeoIntegrationCli.main(NeoIntegrationCli.java:43)

  • Could not log SQL warnings for result set
    net.snowflake.client.jdbc.SnowflakeSQLException: Result set has been closed.
    at net.snowflake.client.jdbc.SnowflakeBaseResultSet.raiseSQLExceptionIfResultSetIsClosed(SnowflakeBaseResultSet.java:74)
    at net.snowflake.client.jdbc.SnowflakeBaseResultSet.getWarnings(SnowflakeBaseResultSet.java:286)
    at sf.util.DatabaseUtility.logSQLWarnings(DatabaseUtility.java:248)
    at schemacrawler.crawl.MetadataResultSet.next(MetadataResultSet.java:485)
    at schemacrawler.crawl.TableColumnRetriever.retrieveColumnsFromMetadata(TableColumnRetriever.java:304)
    at schemacrawler.crawl.TableColumnRetriever.retrieveColumns(TableColumnRetriever.java:107)
    at schemacrawler.crawl.SchemaCrawler.lambda$crawlTables$18(SchemaCrawler.java:603)
    at sf.util.StopWatch.time(StopWatch.java:185)
    at schemacrawler.crawl.SchemaCrawler.crawlTables(SchemaCrawler.java:600)
    at schemacrawler.crawl.SchemaCrawler.crawl(SchemaCrawler.java:817)
    at schemacrawler.utility.SchemaCrawlerUtility.getCatalog(SchemaCrawlerUtility.java:75)
    at org.neo4j.etl.sql.DatabaseClient.getCatalog(DatabaseClient.java:148)
    at org.neo4j.etl.sql.DatabaseCatalog.(DatabaseCatalog.java:81)
    at org.neo4j.etl.sql.metadata.TableInfoAssembler.(TableInfoAssembler.java:35)
    at org.neo4j.etl.commands.DatabaseInspector.buildSchemaExport(DatabaseInspector.java:35)
    at org.neo4j.etl.commands.rdbms.GenerateMetadataMapping.call(GenerateMetadataMapping.java:110)
    at org.neo4j.etl.commands.rdbms.GenerateMetadataMapping.call(GenerateMetadataMapping.java:31)
    at org.neo4j.etl.cli.rdbms.GenerateMetadataMappingCli.run(GenerateMetadataMappingCli.java:118)
    at org.neo4j.etl.util.CliRunner.run(CliRunner.java:42)
    at org.neo4j.etl.util.CliRunner.run(CliRunner.java:35)
    at org.neo4j.etl.NeoIntegrationCli.main(NeoIntegrationCli.java:43)

  • Could not log SQL warnings for result set
    net.snowflake.client.jdbc.SnowflakeSQLException: Result set has been closed.
    at net.snowflake.client.jdbc.SnowflakeBaseResultSet.raiseSQLExceptionIfResultSetIsClosed(SnowflakeBaseResultSet.java:74)
    at net.snowflake.client.jdbc.SnowflakeBaseResultSet.getWarnings(SnowflakeBaseResultSet.java:286)
    at sf.util.DatabaseUtility.logSQLWarnings(DatabaseUtility.java:248)
    at schemacrawler.crawl.MetadataResultSet.next(MetadataResultSet.java:485)
    at schemacrawler.crawl.TableColumnRetriever.retrieveColumnsFromMetadata(TableColumnRetriever.java:304)
    at schemacrawler.crawl.TableColumnRetriever.retrieveColumns(TableColumnRetriever.java:107)
    at schemacrawler.crawl.SchemaCrawler.lambda$crawlTables$18(SchemaCrawler.java:603)
    at sf.util.StopWatch.time(StopWatch.java:185)
    at schemacrawler.crawl.SchemaCrawler.crawlTables(SchemaCrawler.java:600)
    at schemacrawler.crawl.SchemaCrawler.crawl(SchemaCrawler.java:817)
    at schemacrawler.utility.SchemaCrawlerUtility.getCatalog(SchemaCrawlerUtility.java:75)
    at org.neo4j.etl.sql.DatabaseClient.getCatalog(DatabaseClient.java:148)
    at org.neo4j.etl.sql.DatabaseCatalog.(DatabaseCatalog.java:81)
    at org.neo4j.etl.sql.metadata.TableInfoAssembler.(TableInfoAssembler.java:35)
    at org.neo4j.etl.commands.DatabaseInspector.buildSchemaExport(DatabaseInspector.java:35)
    at org.neo4j.etl.commands.rdbms.GenerateMetadataMapping.call(GenerateMetadataMapping.java:110)
    at org.neo4j.etl.commands.rdbms.GenerateMetadataMapping.call(GenerateMetadataMapping.java:31)
    at org.neo4j.etl.cli.rdbms.GenerateMetadataMappingCli.run(GenerateMetadataMappingCli.java:118)
    at org.neo4j.etl.util.CliRunner.run(CliRunner.java:42)
    at org.neo4j.etl.util.CliRunner.run(CliRunner.java:35)
    at org.neo4j.etl.NeoIntegrationCli.main(NeoIntegrationCli.java:43)

  • Could not log SQL warnings for result set
    net.snowflake.client.jdbc.SnowflakeSQLException: Result set has been closed.
    at net.snowflake.client.jdbc.SnowflakeBaseResultSet.raiseSQLExceptionIfResultSetIsClosed(SnowflakeBaseResultSet.java:74)
    at net.snowflake.client.jdbc.SnowflakeBaseResultSet.getWarnings(SnowflakeBaseResultSet.java:286)
    at sf.util.DatabaseUtility.logSQLWarnings(DatabaseUtility.java:248)
    at schemacrawler.crawl.MetadataResultSet.next(MetadataResultSet.java:485)
    at schemacrawler.crawl.TableColumnRetriever.retrieveColumnsFromMetadata(TableColumnRetriever.java:304)
    at schemacrawler.crawl.TableColumnRetriever.retrieveColumns(TableColumnRetriever.java:107)
    at schemacrawler.crawl.SchemaCrawler.lambda$crawlTables$18(SchemaCrawler.java:603)
    at sf.util.StopWatch.time(StopWatch.java:185)
    at schemacrawler.crawl.SchemaCrawler.crawlTables(SchemaCrawler.java:600)
    at schemacrawler.crawl.SchemaCrawler.crawl(SchemaCrawler.java:817)
    at schemacrawler.utility.SchemaCrawlerUtility.getCatalog(SchemaCrawlerUtility.java:75)
    at org.neo4j.etl.sql.DatabaseClient.getCatalog(DatabaseClient.java:148)
    at org.neo4j.etl.sql.DatabaseCatalog.(DatabaseCatalog.java:81)
    at org.neo4j.etl.sql.metadata.TableInfoAssembler.(TableInfoAssembler.java:35)
    at org.neo4j.etl.commands.DatabaseInspector.buildSchemaExport(DatabaseInspector.java:35)
    at org.neo4j.etl.commands.rdbms.GenerateMetadataMapping.call(GenerateMetadataMapping.java:110)
    at org.neo4j.etl.commands.rdbms.GenerateMetadataMapping.call(GenerateMetadataMapping.java:31)
    at org.neo4j.etl.cli.rdbms.GenerateMetadataMappingCli.run(GenerateMetadataMappingCli.java:118)
    at org.neo4j.etl.util.CliRunner.run(CliRunner.java:42)
    at org.neo4j.etl.util.CliRunner.run(CliRunner.java:35)
    at org.neo4j.etl.NeoIntegrationCli.main(NeoIntegrationCli.java:43)

Labels:
0 Kudos
Reply
3 REPLIES 3

davidrsetti
Node Link

‎05-20-2020 08:33 AM

Same issue here.

Is there a way to connect to snowflake and retrieve the metadata?

0 Kudos
Reply

sam_gijare
Graph Buddy
In response to davidrsetti

‎02-09-2021 08:18 PM

Hello David

Alternative Solution:
If you are using AWS cloud then you can eliminate all this by subscribing to other services from Amazon like Glacier or similar services in amazon ecosystems and then retrieve the relevant data from amazon using AWS command Line Interface.

But if you want to deal with snowflakes then it is better to get Professional Services help from Snowflakes CSR.

Thanks
Sameer

0 Kudos
Reply

dominicvivek06
Graph Fellow
In response to davidrsetti

‎02-10-2021 04:24 AM

Hello @davidrsetti , do you want to retrieve the data or the metadata ? My alternative options (my 1st preference ) is data pipeline in python , ETL tools like Talend.

Talend -> Snowflake and Talend: Data quality and governance in the cloud

If you just need metadata, I have created a blog for sql server. There are alot of data dictionary in snowflake (let me know if you need a query), that you can change and retrieve the metadata .

http://www.dominickumar.com/blog/sql-server-database-er-metadata-in-neo4j/

@sam_gijare -> AWS Glacier is an AWS archival storage. Accessing data in and out of Glacier is paid and there is a timeframe for the data availability. You need additional services like AWS glue to crawl, and you dont need additional services for metadata, since snowflake already has data dictionary .

0 Kudos
Reply
Nodes 2022
Nodes
NODES 2022, Neo4j Online Education Summit

All the sessions of the conference are now available online

Watch replays