# Distributed-Graph-Inference-Java
## 1. Introduction
A graph is a structure that can effectively represent objects and the relationships between them. Graph Neural Networks (GNNs) enable deep learning to be applied in the graph domain. However, most GNN models are trained offline and cannot be directly used in real-time monitoring scenarios. In addition, due to the very large data scale of the graph, a single machine cannot meet the demand, and there is a performance bottleneck. Therefore, we propose a distributed graph neural network inference computing framework, which can be applied to GNN models in the form of Encoder-Decoder. We propose the idea of “single-point inference, message passing, distributed computing”, which enables the system to use offline-trained GNNs for real-time inference computations on graph data. In order to maintain the model effect, we add the second-degree subgraph and mailbox mechanism to the continuous iterative calculation. Finally, our results on public datasets show that this method greatly improves the upper limit of inference computation and has better timeliness. And it maintains a good model effect on three types of classical tasks.
## 2. Method
The framework is mainly composed of the following modules: incremental composition, second-degree subgraph calculation, GNN encoder, mailbox, GNN decoder.
## 3. Environment
| Category | Details |
| ------------------- | ---------------------------------------------------- |
| processor | Intel(R) Core(TM) i5-8257U CPU @2.00GHz |
| memory | 16GB |
| OS | macOS Catalina 10.15.7 |
| development tool | IntelliJ IDEA 2020.1.2 |
| running environment | Spark 3.2.0, Hadoop 3.3.0, Scala 2.12.15, Java 1.8.0 |
## 4. Run
You can open the project directly in IDEA, configure the environment, and set the running parameters, then run `Main.java` directly.
### 4.1 Configurable parameters
```shell
Execute command parameter order:
command RESOURCE_PATH DATASET_NAME TASK_NAME CHECKPOINT_FREQUENCY MAX_EVENT_NUM
1. RESOURCE_PATH: Static resource(dataset/model/log/checkpoint/result) path
2. DATASET_NAME: Dataset name(Wikipedia or Reddit)
3. TASK_NAME: The name of the task performed(LP/NC/EC)
4. CHECKPOINT_FREQUENCY: Frequency of truncating RDD lineages(1/2/.../10)
5. MAX_EVENT_NUM: Maximum number of inference events(100/200/500/1000/...)
Execute command parameter example:
command /Users/xxx/project/src/main/resources/ wikipedia LP 3 1500
```
### 4.2 Dataset
[Wikipedia and Reddit](http://snap.stanford.edu/jodie/#datasets)
### 4.3 GNN Model
An example of a model in the path(`./src/main/resources/model/`).
没有合适的资源?快使用搜索试试~ 我知道了~
温馨提示
- 不懂运行,下载完可以私聊问,可远程教学 该资源内项目源码是个人的毕设,代码都测试ok,都是运行成功后才上传资源,答辩评审平均分达到96分,放心下载使用! <项目介绍> 1、该资源内项目代码都经过测试运行成功,功能ok的情况下才上传的,请放心下载使用! 2、本项目适合计算机相关专业(如计科、人工智能、通信工程、自动化、电子信息等)的在校学生、老师或者企业员工下载学习,也适合小白学习进阶,当然也可作为毕设项目、课程设计、作业、项目初期立项演示等。 3、如果基础还行,也可在此代码基础上进行修改,以实现其他功能,也可用于毕设、课设、作业等。 下载后请首先打开README.md文件(如有),仅供学习参考, 切勿用于商业用途。 --------
资源推荐
资源详情
资源评论
收起资源包目录
Distributed-Graph-Inference-Java-master.zip (44个子文件)
Distributed-Graph-Inference-Java-master
pom.xml 3KB
src
main
resources
model
Decoder_wikipedia_LP.pt 248KB
Encoder_wikipedia_LP.pt 1015KB
java
experiment
Infer.java 2KB
dataset
Mail.java 1KB
Edata.java 2KB
GraphX.java 13KB
Vdata.java 4KB
Vfeat.java 1KB
model
DecoderOutput.java 759B
EncoderInput.java 2KB
DecoderInput.java 764B
Encoder.java 2KB
EncoderOutput.java 2KB
DecoderTranslator.java 3KB
EncoderTranslator.java 2KB
Decoder.java 2KB
Main.java 8KB
Test.java 4KB
util
StopWatchExpand.java 2KB
absfunc
vertex
Update2DSubgraph.java 1018B
UpdateHop.java 926B
UpdateEmb.java 1KB
UpdateFeat.java 2KB
UpdateTime.java 1KB
UpdateMailbox.java 1KB
AvgMail.java 681B
edge
MergeEdge.java 665B
UpdateRes.java 3KB
NegFilter.java 847B
triplet
MergeVfeat.java 790B
MergeHop.java 639B
SendHop.java 2KB
MergeEmb.java 843B
MergeMail.java 1KB
SendEmb.java 2KB
SendMail.java 2KB
SendVfeat.java 2KB
config
SparkInit.java 2KB
Constants.java 8KB
shell
remove.sh 261B
META-INF
MANIFEST.MF 43B
.gitignore 168B
README.md 3KB
共 44 条
- 1
资源评论
机器学习的喵
- 粉丝: 711
- 资源: 1296
下载权益
C知道特权
VIP文章
课程特权
开通VIP
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功