WikiData Processing

WikiData Processing

Data Accessing

To download latest-all.json.bz2 on the page https://dumps.wikimedia.org/wikidatawiki/entities/ .

wget is the best tool to download huge file:

wget -c -t 0 https://dumps.wikimedia.org/wikidatawiki/entities/latest-all.json.bz2

Installing Neo4j on Ubuntu

From Repositories (Not recommended)

This way needs supervisor authority, which is inconvenient.

Step 1. (Optional) To install OpenJRE and OpenJDK firstly, if there are no Java runtime environment on the computer.

$ sudo apt-get update
$ sudo apt-get install default-jre
$ sudo apt-get install default-jdk

Step 2. Installing Neo4j.

To use the repository add it to the list of sources:

$ wget -O - https://debian.neo4j.org/neotechnology.gpg.key | sudo apt-key add -
$ echo 'deb https://debian.neo4j.org/repo stable/' | sudo tee /etc/apt/sources.list.d/neo4j.list
$ sudo apt-get update

To install the latest Neo4j Community Edition:

$ sudo apt-get install neo4j

User Installation

To extract files from neo4j-community-3.5.8-unix.tar.gz

$ tar zxvf neo4j-community-3.5.8-unix.tar.gz

Adding follow lines to ~/.bashrc, and 'source' it.

# neo4j
export NEO4J_HOME="/home/fyb/neo4j-community-3.5.8"
export PATH=$PATH:$NEO4J_HOME/bin

To open remote accessing authority.

$ vi neo4j-community-3.5.8/conf/neo4j.conf

change #dbms.connector.http.listen_address=:7474
to dbms.connector.http.listen_address=0.0.0.0:7474

change #dbms.connector.bolt.listen_address=:7687
to dbms.connector.bolt.listen_address=0.0.0.0:7687

To start service, <NEO4J_HOME> is the top level directory referred to neo4j-community-3.5.8

$ neo4j console

Installing NodeJs

Node.js v12.x:

$ curl -sL https://deb.nodesource.com/setup_12.x | sudo -E bash -
$ sudo apt-get install -y nodejs

Importing

We employ wikidata-neo4j-importer to import wikidata into neo4J.

https://github.com/findie/wikidata-neo4j-importer

some requirements need to be installed.

$ npm install neo4j-driver async n-readlines cli-color slugify

Now, we can run index.js to import WikiData into neo4j !

$ node index.js

Bibliography

http://debian.neo4j.org/

https://github.com/nodesource/distributions/blob/master/README.md#deb

https://neo4j.com/download-thanks/?edition=community&release=3.5.8&flavour=unix