About to jump off cliff.. mongo-connector still cannot connect to neo4j

[john_allen]

github.com/yougov/mongo-connector

Guys.. all fixed after 2 days and neo4j auth disabled etc and STILL get a py2neo.packages.httpstream...

opened 04:46PM - 22 Oct 20 UTC

johnda98

about to jump off a cliff everythings cool.. mongo.. rep set.. neo desktop uip.. browser up.. db started.. auth disabled on neo4j...

about to jump off a cliff

everythings cool.. mongo.. rep set.. neo desktop uip.. browser up.. db started.. auth disabled on neo4j conf ... but still mongo-connector can't hook to the neo browser and dump my rep set in there

any clues guys ??

leveridge@leveridge-PowerEdge-R710:~$ mongo-connector -m localhost:27017 -t http://localhost:7474/db/data -d neo4j_doc_manager
Logging to /home/leveridge/mongo-connector.log.

.. in the log..

py2neo.packages.httpstream.http.ClientError: 401 Unauthorized

its crazy.. no darn reason.

using py3 and still have py2 also of course as it uses some libs from there.. but installed mongo-connector via pip3 and neo4j-doc-manager via pip3 all coolio

set the neo conf to false on the authentication.. thats cool.. re-started instance.. all cool. and STILL mongo connect cant connect to my neo

welcome ANY clues thanks

mongo 4.4 .. yep Rep set all coolio in the mongo conf.. its happy . py3 good.. neo is 4.1

.. log file.. from mongo-connect

Traceback (most recent call last):
File "/home/leveridge/.local/lib/python3.6/site-packages/py2neo/core.py", line 258, in get
response = self.__base.get(headers=headers, redirect_limit=redirect_limit, **kwargs)
File "/home/leveridge/.local/lib/python3.6/site-packages/py2neo/packages/httpstream/http.py", line 966, in get
return self.__get_or_head("GET", if_modified_since, headers, redirect_limit, **kwargs)
File "/home/leveridge/.local/lib/python3.6/site-packages/py2neo/packages/httpstream/http.py", line 943, in __get_or_head
return rq.submit(redirect_limit=redirect_limit, **kwargs)
File "/home/leveridge/.local/lib/python3.6/site-packages/py2neo/packages/httpstream/http.py", line 452, in submit
return Response.wrap(http, uri, self, rs, **response_kwargs)
File "/home/leveridge/.local/lib/python3.6/site-packages/py2neo/packages/httpstream/http.py", line 489, in wrap
raise inst
py2neo.packages.httpstream.http.ClientError: 401 Unauthorized

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/leveridge/.local/lib/python3.6/site-packages/mongo_connector/util.py", line 33, in wrapped
return f(*args, **kwargs)
File "/home/leveridge/.local/lib/python3.6/site-packages/mongo_connector/doc_managers/neo4j_doc_manager.py", line 78, in bulk_upsert
tx = self.graph.cypher.begin()
File "/home/leveridge/.local/lib/python3.6/site-packages/py2neo/core.py", line 661, in cypher
metadata = self.resource.metadata
File "/home/leveridge/.local/lib/python3.6/site-packages/py2neo/core.py", line 213, in metadata
self.get()
File "/home/leveridge/.local/lib/python3.6/site-packages/py2neo/core.py", line 261, in get
raise Unauthorized(self.uri.string)
py2neo.error.Unauthorized: http://localhost:7474/db/data/

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/leveridge/.local/lib/python3.6/site-packages/mongo_connector/oplog_manager.py", line 668, in do_dump
upsert_all(dm)
File "/home/leveridge/.local/lib/python3.6/site-packages/mongo_connector/oplog_manager.py", line 651, in upsert_all
dm.bulk_upsert(docs_to_dump(from_coll), mapped_ns, long_ts)
File "/home/leveridge/.local/lib/python3.6/site-packages/mongo_connector/util.py", line 44, in wrapped
raise new_type(str(exc_value)).with_traceback(exc_tb)
File "/home/leveridge/.local/lib/python3.6/site-packages/mongo_connector/util.py", line 33, in wrapped
return f(*args, **kwargs)
File "/home/leveridge/.local/lib/python3.6/site-packages/mongo_connector/doc_managers/neo4j_doc_manager.py", line 78, in bulk_upsert
tx = self.graph.cypher.begin()
File "/home/leveridge/.local/lib/python3.6/site-packages/py2neo/core.py", line 661, in cypher
metadata = self.resource.metadata
File "/home/leveridge/.local/lib/python3.6/site-packages/py2neo/core.py", line 213, in metadata
self.get()
File "/home/leveridge/.local/lib/python3.6/site-packages/py2neo/core.py", line 261, in get
raise Unauthorized(self.uri.string)
mongo_connector.doc_managers.error_handler.Neo4jOperationFailed: http://localhost:7474/db/data/
2020-10-22 11:41:02,865 [ERROR] mongo_connector.oplog_manager:723 - OplogThread: Failed during dump collection cannot recover! Collection(Database(MongoClient(host=['localhost:27017'], document_class=dict, tz_aware=False, connect=True, replicaset='replica01'), 'local'), 'oplog.rs')
2020-10-22 11:41:03,855 [ERROR] mongo_connector.connector:411 - MongoConnector: OplogThread <OplogThread(Thread-2, started 140038306244352)> unexpectedly stopped! Shutting down

soooo..

turned off ie true the neo4j conf dbms and set the ENV variables to 'neo4j' and 'password' set in my database in neo4j desktop (which has same name as replicaset 'replica01' ..

try mongo-connector again and now i get a py2neo '404'

"2020-10-22 14:55:18,169 [ALWAYS] mongo_connector.connector:50 - Starting mongo-connector version: 3.1.1
2020-10-22 14:55:18,170 [ALWAYS] mongo_connector.connector:50 - Python version: 3.6.9 (default, Oct 8 2020, 12:12:24)
[GCC 8.4.0]
2020-10-22 14:55:18,171 [ALWAYS] mongo_connector.connector:50 - Platform: Linux-4.15.0-122-generic-x86_64-with-Ubuntu-18.04-bionic
2020-10-22 14:55:18,171 [ALWAYS] mongo_connector.connector:50 - pymongo version: 3.11.0
2020-10-22 14:55:18,179 [ALWAYS] mongo_connector.connector:50 - Source MongoDB version: 4.4.1
2020-10-22 14:55:18,180 [ALWAYS] mongo_connector.connector:50 - Target DocManager: mongo_connector.doc_managers.neo4j_doc_manager version: unknown
2020-10-22 14:55:18,223 [CRITICAL] mongo_connector.oplog_manager:714 - Exception during collection dump
Traceback (most recent call last):
File "/home/leveridge/.local/lib/python3.6/site-packages/py2neo/core.py", line 258, in get
response = self.__base.get(headers=headers, redirect_limit=redirect_limit, **kwargs)
File "/home/leveridge/.local/lib/python3.6/site-packages/py2neo/packages/httpstream/http.py", line 966, in get
return self.__get_or_head("GET", if_modified_since, headers, redirect_limit, **kwargs)
File "/home/leveridge/.local/lib/python3.6/site-packages/py2neo/packages/httpstream/http.py", line 943, in __get_or_head
return rq.submit(redirect_limit=redirect_limit, **kwargs)
File "/home/leveridge/.local/lib/python3.6/site-packages/py2neo/packages/httpstream/http.py", line 452, in submit
return Response.wrap(http, uri, self, rs, **response_kwargs)
File "/home/leveridge/.local/lib/python3.6/site-packages/py2neo/packages/httpstream/http.py", line 489, in wrap
raise inst
py2neo.packages.httpstream.http.ClientError: 404 Not Found

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/leveridge/.local/lib/python3.6/site-packages/mongo_connector/util.py", line 33, in wrapped
return f(*args, **kwargs)
File "/home/leveridge/.local/lib/python3.6/site-packages/mongo_connector/doc_managers/neo4j_doc_manager.py", line 78, in bulk_upsert
tx = self.graph.cypher.begin()
File "/home/leveridge/.local/lib/python3.6/site-packages/py2neo/core.py", line 661, in cypher
metadata = self.resource.metadata
File "/home/leveridge/.local/lib/python3.6/site-packages/py2neo/core.py", line 213, in metadata
self.get()
File "/home/leveridge/.local/lib/python3.6/site-packages/py2neo/core.py", line 267, in get
raise_from(self.error_class(message, **content), error)
File "/home/leveridge/.local/lib/python3.6/site-packages/py2neo/util.py", line 235, in raise_from
raise exception
py2neo.error.GraphError: HTTP GET returned response 404

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/leveridge/.local/lib/python3.6/site-packages/mongo_connector/oplog_manager.py", line 668, in do_dump
upsert_all(dm)
File "/home/leveridge/.local/lib/python3.6/site-packages/mongo_connector/oplog_manager.py", line 651, in upsert_all
dm.bulk_upsert(docs_to_dump(from_coll), mapped_ns, long_ts)
File "/home/leveridge/.local/lib/python3.6/site-packages/mongo_connector/util.py", line 44, in wrapped
raise new_type(str(exc_value)).with_traceback(exc_tb)
File "/home/leveridge/.local/lib/python3.6/site-packages/mongo_connector/util.py", line 33, in wrapped
return f(*args, **kwargs)
File "/home/leveridge/.local/lib/python3.6/site-packages/mongo_connector/doc_managers/neo4j_doc_manager.py", line 78, in bulk_upsert
tx = self.graph.cypher.begin()
File "/home/leveridge/.local/lib/python3.6/site-packages/py2neo/core.py", line 661, in cypher
metadata = self.resource.metadata
File "/home/leveridge/.local/lib/python3.6/site-packages/py2neo/core.py", line 213, in metadata
self.get()
File "/home/leveridge/.local/lib/python3.6/site-packages/py2neo/core.py", line 267, in get
raise_from(self.error_class(message, **content), error)
File "/home/leveridge/.local/lib/python3.6/site-packages/py2neo/util.py", line 235, in raise_from
raise exception
mongo_connector.doc_managers.error_handler.Neo4jOperationFailed: HTTP GET returned response 404
2020-10-22 14:55:18,225 [ERROR] mongo_connector.oplog_manager:723 - OplogThread: Failed during dump collection cannot recover! Collection(Database(MongoClient(host=['localhost:27017'], document_class=dict, tz_aware=False, connect=True, replicaset='replica01'), 'local'), 'oplog.rs')
2020-10-22 14:55:19,197 [ERROR] mongo_connector.connector:411 - MongoConnector: OplogThread <OplogThread(Thread-2, started 139719637624576)> unexpectedly stopped! Shutting down"