# SADGA
``` bash
mkdir -p dataset third_party
```
Download the dataset: [Spider](https://yale-lily.github.io/spider). Then unzip `spider.zip` into the directory `dataset`.
```
└── dataset
├── database
│ ├── academic
│ │ ├──academic.sqlite
│ │ ├──schema.sql
│ ├── ...
├── dev_gold.sql
├── dev.json
├── README.txt
├── tables.json
├── train_gold.sql
├── train_others.json
└── train_spider.json
```
Download and unzip [Stanford CoreNLP](https://download.cs.stanford.edu/nlp/software/stanford-corenlp-full-2018-10-05.zip) to the directory `third_party`. Note that this repository requires a JVM to run it.
```
└── third_party
└── stanford-corenlp-full-2018-10-05
├── ...
```
### Create environment
We trained our models on one server with a single NVIDIA GTX 3090 GPU with 24GB GPU memory. In our experiments, we use **python 3.7**, **torch 1.7.1** with **CUDA version 11.0**. We create conda environment `sadgasql`:
```bash
conda create -n sadgasql python=3.7
source activate sadgasql
pip install torch==1.7.1+cu110 -f https://download.pytorch.org/whl/torch_stable.html
pip install -r requirements.txt
python -c "import nltk; nltk.download('stopwords'); nltk.download('punkt')"
```
### Run
All configs of the experiments and models are in the files `sadga-bert-run.jsonnet`, `sadga-roberta-run.jsonnet`.
##### Step 1. Preprocess
```bash
python run.py --mode preprocess --config sadga-[roberta|bert]-run.jsonnet
```
##### Step 2. Training
```bash
python run.py --mode train --config sadga-[roberta|bert]-run.jsonnet
```
- After the training, we can obtain some model-checkpoints in the directory `{logdir}/{model_name}/`, e.g., `sadga_roberta_bs=6_lr=2.4e-04_blr=3.0e-06/model_checkpoint-00000060`.
##### Step 3. Inference
```bash
python run.py --mode infer --config sadga-[roberta|bert]-run.jsonnet
```
##### Step 4. Eval
```bash
python run.py --mode eval --config sadga-[roberta|bert]-run.jsonnet
```
You can download the logdir result directory from the link:[logdir](https://drive.google.com/file/d/1KDizYvhEliAgfiGsFopIF_FxL32AGRbO/view?usp=sharing) . Please download it and run it if you are having issue with running the above 4 steps. Add the logdir to the root folder.
##### Step 5. Cosine Similarity
```bash
python cosineSimilarity.py
```
使用 Graph Networks 与 BERT 和 Roberta 进行文本到 SQL 泛化.zip
版权申诉
8 浏览量
2024-04-23
12:23:02
上传
评论
收藏 273KB ZIP 举报
使用 Graph Networks 与 BERT 和 Roberta 进行文本到 SQL 泛化.zip （110个子文件） 
Spider_f1.asdl 4KB Spider_f2.asdl 4KB Spider.asdl 4KB .gitignore 64B sadga-bert-config.jsonnet 3KB sadga-roberta-config.jsonnet 3KB sadga-roberta-run.jsonnet 348B sadga-bert-run.jsonnet 342B README.md 2KB README.md 62B encoder.py 50KB decoder.py 35KB evaluation.py 30KB evaluation.py 30KB spider.py 26KB spider_enc_modules.py 20KB spider_beam_search.py 17KB process_sql.py 16KB process_sql.py 16KB tree_traversal.py 14KB rat.py 12KB variational_lstm.py 12KB sadga.py 11KB ast_util.py 10KB batched_sequence.py 9KB spider.py 7KB infer_tree_traversal.py 7KB train.py 6KB cosineSimilarity.py 6KB spider_match_utils.py 6KB infer.py 5KB attention.py 5KB saver.py 5KB optimizers.py 4KB train_tree_traversal.py 4KB enc_dec.py 3KB vocab.py 3KB pretrained_embeddings.py 3KB gated_graph_conv.py 3KB spider_dec_func.py 3KB run.py 3KB beam_search.py 2KB random_state.py 2KB registry.py 2KB corenlp.py 2KB eval.py 2KB preprocess.py 2KB abstract_preproc.py 1KB evaluation.py 1KB indexed_file.py 1KB serialization.py 555B __init__.py 97B __init__.py 44B __init__.py 36B __init__.py 21B __init__.py 21B __init__.py 21B __init__.py 21B __init__.py 0B __init__.py 0B __init__.py 0B encoder.cpython-37.pyc 37KB evaluation.cpython-37.pyc 25KB evaluation.cpython-37.pyc 24KB decoder.cpython-37.pyc 21KB spider.cpython-37.pyc 18KB spider_enc_modules.cpython-37.pyc 15KB process_sql.cpython-37.pyc 12KB process_sql.cpython-37.pyc 12KB sadga.cpython-37.pyc 10KB tree_traversal.cpython-37.pyc 10KB rat.cpython-37.pyc 10KB batched_sequence.cpython-37.pyc 9KB ast_util.cpython-37.pyc 8KB spider_beam_search.cpython-37.pyc 8KB variational_lstm.cpython-37.pyc 7KB spider.cpython-37.pyc 6KB infer_tree_traversal.cpython-37.pyc 6KB train.cpython-37.pyc 5KB pretrained_embeddings.cpython-37.pyc 5KB attention.cpython-37.pyc 5KB vocab.cpython-37.pyc 5KB saver.cpython-37.pyc 4KB spider_match_utils.cpython-37.pyc 4KB enc_dec.cpython-37.pyc 4KB infer.cpython-37.pyc 4KB optimizers.cpython-37.pyc 4KB train_tree_traversal.cpython-37.pyc 4KB gated_graph_conv.cpython-37.pyc 3KB spider_dec_func.cpython-37.pyc 2KB random_state.cpython-37.pyc 2KB abstract_preproc.cpython-37.pyc 2KB registry.cpython-37.pyc 2KB corenlp.cpython-37.pyc 2KB preprocess.cpython-37.pyc 2KB beam_search.cpython-37.pyc 1KB eval.cpython-37.pyc 1KB evaluation.cpython-37.pyc 1KB serialization.cpython-37.pyc 1KB __init__.cpython-37.pyc 303B共 110 条
- 1
- 2
资源评论
