Within hours of saying the largest tables had 10,000 rows, I posted a dataset with nearly 400 thousand rows. Sigh. 
Offerings of Data #3
3. North Pacific Salmon Abundance and Biomass Data
From Dr. Jim Irvine, Pacific Biological Station: "Scott, thanks for distributing these data. ... these data are relatively simple, on the other hand, if you ever wondered how many salmon there are in the Pacific Ocean, and how this has changed since 1925, this is where to find out. There are lots of tables but the graphs in our paper will give you a better feel for the data." PAPER.
"I think the [ISDL] folks would be interested in seeing differences between Alaska and the southern states in terms of salmon numbers and biomass. The various time series, if spatially linked to maps, would be relatively easy to show. The NPAFC data [see following post] have the advantage of including catches for all salmon species by area. The Ruggerone and Irvine data are only 3 species but have the advantage of separating hatchery from wild fish as well as numbers vs adult biomass vs adult plus immature biomass, by area."
This paper was featured in a magazine article with tableau figures that allow readers to play with the data.
LINK TO DATA Please acknowledge the scientists in any use of these data.
format: .xlsx, 21 tables (multiple per sheet), 64 rows of 15 variables.
metadata: extensive, as sheets in this .xlsx file.
source: reviewed science publication, see link above
contacts: Dr. Gregory T. Ruggerone, Natural Resources Consultants Inc., Suite 404, 4039 21st Avenue West, Seattle, WA, 98199, USA.GRuggerone@nrccorp.com
Dr. James R. Irvine, Fisheries and Oceans Canada, Pacific Biological Station, 3190 Hammond Bay Road, Nanaimo, BC, V9T 6N7, Canada. James.Irvine@dfo-mpo.gc.ca
I will start defining the Darwin Core Simple Model in the Salmon Data Laboratory Graph and attach edges to show how the data may be related. I'll then ingest some of the attach data into a real graph and share that out to show how the data can be fit into the Darwin model. I may get some of the biological terms wrong to start, but it'll be a good learning experience and nothing that can't be remedied with a simple update. Up to this point, salmon for me was something on a menu; not something with Latin and Greek names. 
Thanks! Our first graph. See figure below. More to this than meets the eye. Extensive properties for each "resource" (type of node). That was a lot of work.
"Come Watson! The game is afoot."
Now I /we have to figure out how to create a lot of instances of each resource, so:
- Created Google Sheets corresponding to this schema, HERE.
- Wrote an R script to create (empty) data.frames for this schema, HERE
Darwin Core is for museum catalogues, not for organizing salmon datasets, practices, analyses, workflows, data products,. And the surrounding people, projects,. But:
- we do not need to specify every field in each resource.
- we must add new resources.
Obviously Person, Activity, Organization),.
Less so: WorkFlow, Model, DataSet, Practice (?) - Helpful to future users if we predefine layers (ontologies,networks) such as Place-[In]-Place, Organization-[In]-Organization, Practice-[in]-Practice.
I conclude we are going to have to build our own Salmon Ontology. It will not be small.
This needs to be a group grope, but turning into a standards committee is a trap. We need "good enough to proceed." The Nazi Librarian ("no data for you!") will have her/his/its day.
Another Google Sheet for that. Stand by!
International Year of Salmon (IYS) Workshop on Salmon Status and Trends
2019-01-21/22, Vancouver BC Canada
Problem:
In recent decades, the productivity of salmon has become increasingly uncertain with many populations experiencing extremes related to timing, abundance and size with serious social, economic and conservation impacts. Much has been learned in previous workshops but a lack of consistency in approaches to categorize biological status and trends, terminology to indicate status, requirements and standards for different types of data, spatial and temporal scales for comparison and aggregation, and ways of communicating findings significantly impedes the timeliness, efficiency and effectiveness of scientific investigations. Our data systems do not match our technological capacity and social/scientific inquiry needs. Many agencies have a commitment to open data but are challenged to achieve it given the significant costs associated with bringing historic data online. Increasing variability in a rapidly changing environment demands rapid access to integrated data for comparative and mechanistic studies of the distribution and productivity of salmon across life history stages and associated eco- regions.
Solution: (extracts thereof)
The primary goal of this workshop will be to identify a series of legacy datasets and standards associated with major categories of data. These datasets will be the focus of subsequent analytical workshops; additional workshops may concentrate on communication of scientific results. Ideally these separate but linked workshops will [cycle and grow]* over the course of IYS.
* but not yet funded.
Complete announcement HERE..
A search and rescue team for salmon data
https://www.nceas.ucsb.edu/news/a-search-and-rescue-team-for-salmon-data
I wrote to Matt Jones, a database guru at the National Centre for Ecological Analysis and Synthesis (NCEAS), introducing ISDL and asking for advice. Specifically about compliance with Darwin Core protocols and ontology as we build knowledge graphs and analyses to share across the environmental sector. ISDL has much to learn from the NCEAS project Southern Alaska Salmon and People or SASAP.
NPAFC catch data. You can search for "NPAFC" to get a list all of the available public graphs.
The beauty of graphs. This is the hatchery release data for US and Canada, NPAFC Hatchery Statistics Species Data - Canada, United States, graph 2b1841ac-73d1-4603-9492-2f281f375171.
Inspired by Mike Bostock's population pyramid, global salmon hatchery release data plotted as an interactive pyramid. https://isdl-220916.appspot.com/release
You might imagine my excitement upon discovering that the Instituut Natuur-en Bosonderzoek in Brussels -- Institute for Nature and Forest Research -- records fish counts back to 1753! Alas, only 3 observations of Atlantic salmon: 2 in 2012, 1 in 2017. Which seems rather odd. 
WorkFlow, finally!
Extensive search revealed several promising workflow packages, but either (A) hopeless as tools for persons other than professional computer programmers/sysadmins, or (B) not being maintained and, frankly, old-fashioned. Australia to the rescue, thank-you CSIRO, for WorkSpace.
An excellent GUI for building workflows, and, to my delight, nested workflows. "Nesting does make working with complex workflow arrangements significantly easier."
I have pounced on this like a Velociraptor on a tourist.
Thanks for keeping everyone in the community updated on this projects' updates. This is awesome!!!! It's really cool to be able to see the process as it moves along. 
The important news is that Fisheries and Oceans Canada implemented a neo4j graph database as fully developed Enterprie IT. Their Azure Cloud, with the DB and microservices in Docker Containers. Tools include NLP and AI for dissecting, loading, and linking messy Excel spreadsheet, Postal Codes, GeoNames.org, First Nations locations,.
As part of the Fisheries and Oceans experiment with neo4j knowledge graph, I insisted, not entirely reasonably, on R+Shiny as platform for a user-friendly (no code) UI because that will allow the salmon research and management community to expand the UI themselves, adding more tools, Views, reports,.
Please see the silent two-minute demo showing GraphView that allows end-users to add and edit content, and showing ability to flip between TableView, MapView, and GraphView. Thank you, George Batten of Sitka Technologies in Portland, for an excellent Proof of Concept.
This system is a knowledge graph for fish, so we called it Graphish (and ourselves Graphishtas). So the UI is called Graphishui and, no, not connected to feng shui.
NODES 2022, Neo4j Online Education Summit
All the sessions of the conference are now available online
