Golang Driver Sessions are slow compared to creating multiple Drivers

[ferdz_steenkamp]

Hi, this is quite weird, my colleague showed me a benchmark difference that is absolutely massive and makes no sense to us. We'd like to find out if we're doing something wrong or if this is a bug in our code?

Basically, we read a list of 25 userID's and run this query:

func userTx(db neo4j.Driver, userID string) {
	session := db.NewSession(neo4j.SessionConfig{
		AccessMode: neo4j.AccessModeRead,
	})
	defer session.Close()

	session.ReadTransaction(func(tx neo4j.Transaction) (any, error) {
		result, err := tx.Run(`
		MATCH(u:User)-[:HAS]->(:Voucher)-[:IS_TYPE]->(:Type)-[:SPENDS_AT]->(m:Merchant)
			WHERE 	u.UserID = $UserID

			RETURN m LIMIT 1
		`, map[string]any{
			"UserID": userID,
		})

		u.PanicIfErr(err)

		record, err := result.Single()
		u.PanicIfErr(err)

                printRecord(record)
                return record, err
	})
}

We are querying against a Free Aura instance in the UK (from South Africa) meaning that latency is expected. Running this query against 25 users clocks in at roughly 18.5 seconds as such:

driver, err := neo4j.NewDriver(uri, auth)
u.PanicIfErr(err)
defer driver.Close()
	
UserRecords := 25
records := readCsvFile("Users.csv")

var wgA sync.WaitGroup
xA := time.Now()

for i := 1; i < UserRecords; i++ {
	wgA.Add(1)
	go func(XXX int) {
		defer wgA.Done()
                    userTx(driver, records[XXX][0])
		}(i)
	}
wgA.Wait()
fmt.Printf("Time with single driver %s", time.Since(xA))

It feels like transactions are running in sync. But if we defer creating the driver until we're in the go routine (not recommended, I know), we get it down to 1.4 seconds! This is a huge difference. Are we doing something in the above example that's preventing full parallelism or are we perhaps misunderstanding our benchmark somehow?

func userTxCreateDriver(userID string) {
	driver, err := neo4j.NewDriver(uri, auth)
	u.PanicIfErr(err)
	defer driver.Close()

	session := driver.NewSession(neo4j.SessionConfig{
		AccessMode: neo4j.AccessModeRead,
	})
	defer session.Close()

	result, err := session.ReadTransaction(func(tx neo4j.Transaction) (any, error) {
		result, err := tx.Run(`
			MATCH(u:User)-[:HAS]->(:Voucher)-[:IS_TYPE]->(:Type)-[:SPENDS_AT]->(m:Merchant) 
			WHERE 	u.UserID = $UserID
			
			RETURN m LIMIT 1
		`, map[string]any{
			"UserID": userID,
		})

                u.PanicIfErr(err)
		record, err := result.Single()
		u.PanicIfErr(err)

		printRecord(record)

		return record, err
	})
}

Any help on this would be appreciated, this is literally a difference of being able to use Neo4j Aura or not. Thanks!

Driver -> "github.com/neo4j/neo4j-go-driver/v4/neo4j"

Golang -> go version go1.18.3 darwin/arm64

[ferdz_steenkamp · Accepted solution]

Figured it out, it's the time it takes for the driver to build up a larger connection pool, running the same test again immediately gives an expected result of about 600ms. Was just a faulty benchmark 🙂