Neo4j

neo4joe · ‎08-21-2019

I am doing some dev-ops work in the AWS environment (unmanaged) with Neo4j Enterprise causal cluster using version 3.5.8. I would like to be able to add a new user from an Ubuntu bash command line. The best resource I have found so far is here:

If there is a plugin or something I can add to the environment to enable this, that's fine. And if there is something in Neo4j's roadmap, that is interesting too. I didn't see anything in the change log for the future Neo4j 4.0

david_allen · ‎08-21-2019

No plugin or web API necessary

If you have neo4j installed on a machine, you'll get the cypher-shell command.

You can use cypher-shell as part of bash scripts or any other automation approach to execute queries against a Neo4j cluster. Make sure to use bolt+routing://my-neo4j-endpoint:7687 to ensure that you're always talking to the cluster, and not just one individual node.

Finally to create a user -- docs here:

CALL dbms.security.createUser('jane', 'abracadabra', true)

https://neo4j.com/docs/operations-manual/current/tutorial/role-based-access-control/

Putting it all together, something like this:

export NEO4J_PASSWORD=secret
export NEO4J_USERNAME=neo4j
export ENDPOINT=bolt+routing://my-cluster:7687

echo "CALL dbms.security.createUser('jane', 'abracadabra', true);" | cypher-shell -a $ENDPOINT -u $NEO4J_USERNAME -p $NEO4J_PASSWORD

mojo2go · ‎09-24-2019

Thanks for showing how to put this together! I have been playing with the technique, and have found a way to create a user and set that user's native role in a single line. (Expressing it all in a single line seems to be really important to our dev ops guy.)

export NEO4J_USERNAME=neo4j;
export NEO4J_PASSWORD=neo;
export ENDPOINT="bolt+routing://localhost:7688";
export NEW_USER_USERNAME="'eddy'"; 
export NEW_USER_PASSWORD="'eddypassword'";
echo "CALL dbms.security.createUser($NEW_USER_USERNAME,$NEW_USER_PASSWORD,true) WITH * CALL dbms.security.addRoleToUser('editor',$NEW_USER_USERNAME) RETURN 'ok';" | ./bin/cypher-shell -a $ENDPOINT -u $NEO4J_USERNAME -p $NEO4J_PASSWORD

But the limitation I have found is with specifying the endpoint. I have 3 members in a causal cluster. I feel that the "bolt+routing" part should find the right server regardless of which of those three servesr I name.
At the moment 7688 is my 'write' server. If I specify that one it works; it creates a new user. But if I specify 7689 or 7690, I get an error message: "Could not perform discovery. No routing servers available." The problem is that this script can't drop down into cypher-shell. It just has to pass in parameter.

Shouldn't it automatically find the write server?

I am testing this in a Docker container on Ubuntu, also in a Docker container on OSX. Both environments work the same.

david_allen · ‎09-25-2019

It should automatically find the write server irrespective of what port you connect to. But this in turn requires that each of your cluster nodes running in Docker have the correct default_advertised_address setting, which is what gets published to your client, and that you've verified causal clustering is working correctly.

When you connect to any node in the cluster, you can run CALL dbms.cluster.overview() to see roughly what the client sees. If that is missing any entry or is advertising the wrong addresses, it means you have a configuration issue with your cluster.

mojo2go · ‎09-25-2019

The servers can see one another, and changes to the graph do propegate, but changes to users don't propagate.

After creating a couple of new users on the leader, I point the Neo4j Browser for each of the three servers, and run two commands:

CALL dbms.cluster.overview()
and
CALL dbms.security.listUsers

The first report is identical across all three servers, and correctly shows the other servers of the cluster and all their roles.

The second report is different on the leader; it shows the neo4j user, plus the two users I created. Whereas the other two servers only show the neo4j user. Apparently users don't propagate across the cluster, as explained here: https://neo4j.com/docs/operations-manual/current/authentication-authorization/native-user-role-manag...
so immediately after I create a new user + role I will need to manually copy these three folders to the non-leader servers.

/var/lib/neo4j/data/dbms/auth
/var/lib/neo4j/data/dbms/
/var/lib/neo4j/data/dbms/

So propagation works for data, but not for users. Is that correct? If so must I shut down those servers when I update them?

And just to turn over every stone..the link above discusses native users. Is there such a thing as a non-native user for Neo4j?

dana_canzano · ‎09-26-2019

correct users/roles do not propogate in a causal cluster automatically. See

https://neo4j.com/docs/operations-manual/3.5/authentication-authorization/native-user-role-managemen...

and also

https://github.com/craigtaverner/neo4j-cluster-rsync-auth as a option to rsync these files

mojo2go · ‎09-26-2019

Thank you Dana.
Can you confirm two more points around routing for me?

The routing protocol ( bolt+routing) works only in a cluster setup. It will fail on a single instance setup. If I am on a single instance I have to drop the +routing part of the protocol.
Routing in a cluster only works from the leader to followers, not from followers to leader. Followers know who the other members are, including leader, but if you write to a follower it will not pass that request to the leader.

The above is what I am experiencing. If I am wrong on either then I will continue investigating.

dana_canzano · ‎09-26-2019

yes. today bolt+routing:// will fail on a single instance non-clustered Neo4j setup.
a WRITE sent to a follower will foward it to the LEADER if
a. you connect with bolt+routing://
and
b. you have defined the transaction as a WRITE transaction, for example via session.WriteTransaction https://neo4j.com/docs/driver-manual/current/sessions-transactions/#driver-transactions-transaction-...

mojo2go · ‎09-27-2019

Two more routing related questions.

When writing queries from Neo4j Desktop does Neo4j Desktop work out what the cypher is doing and then wrap the request in the correct Transaction type (read or write)?
I managed to use the session.WriteTransaction in a Python script. Is it possible to specify 'session.WriteTransaction' in a cypher-shell command, or from a bash prompt, or is it necessary for that to come from one of the language API's java/go/c++/python/javascript..?

dana_canzano · ‎09-29-2019

Neo4j Browser is a Neo4j Bolt Javascript enabled client and cypher-shell is a Neo4j Bolt Java enabled client. As each does not allow you to specify whether the cypher is a READ or WRITE, it must send each as a WRITE so as not to result in error. Thus all cypher statements will go to the leader.

mojo2go · ‎09-29-2019

Okay, that's useful to know, so now I understand why the browser and cypher-shell just work. I don't have to worry about sending a session.WriteTransaction along with my cypher-shell request because that decision has already been made for me, it's already wrapped in one.

It's also good to know that when I send a session.WriteTransaction it doesn't specifically mean that the cypher has to execute a CRUD operation; it just means route this to a (the) server that can write. If it doesn't update something, no biggie.

And finally, by deduction, the only way I can send a session.ReadTransaction is via one of the bolt language drivers (https://neo4j.com/developer/language-guides/#neo4j-drivers) or something that uses one of those.

Thanks @david.allen & @dana.canzano for helping me navigate causal cluster routing and user management. This thread has cleared up a lot of vague questions I've had around routing, I expect it will be useful to others too, particularly those not using LDAP.

Neo4j

Create new user from Ubuntu command line