基于Python语言的MinerU设计源码从GitHub导入

共292个文件

py：167个

json：51个

md：21个

版权申诉

Python

GitHub

源码

3 浏览量 2024-10-05 20:04:14 上传评论收藏 56.6MB ZIP 举报

资源推荐

资源详情

资源评论

收起资源包目录

基于Python语言的MinerU设计源码从GitHub导入（292个子文件）

__init__py 0B

README.md.bak 11KB

README_zh-CN.md.bak 11KB

lid.176.ftz 916KB

.gitignore 373B

para_test_pdf_ids.ini 14KB

preproc_out.json 7.88MB

preproc_out.json 7.84MB

preproc_out.json 3.53MB

preproc_out.json 2.69MB

preproc_out.json 2.52MB

preproc_out.json 1.46MB

preproc_out.json 1.17MB

preproc_out.json 1.09MB

preproc_out.json 723KB

preproc_out.json 664KB

preproc_out.json 570KB

preproc_out.json 538KB

preproc_out.json 500KB

preproc_out.json 484KB

preproc_out.json 276KB

preproc_out.json 275KB

preproc_out.json 245KB

paper_recogPara.json 233KB

pdf_dic.json 214KB

preproc_out.json 203KB

preproc_out.json 190KB

demo1.json 187KB

preproc_out.json 182KB

preproc_out.json 166KB

vertical_en_blocks.json 165KB

preproc_out.json 164KB

academic_literature_0b2c9c91f5232541a7ace8984df306b2_model.json 150KB

preproc_out.json 148KB

preproc_out.json 129KB

preproc_out.json 126KB

demo2.json 113KB

test_metascan_classify_data.json 104KB

preproc_out.json 80KB

pymu_textblocks.json 70KB

academic_literature_fbdb99151e811688574c0c4c67341074_model.json 63KB

preproc_out.json 61KB

academic_literature_f7904bc37cc2e25c1e3e412978854b10_model.json 58KB

preproc_out.json 49KB

research_report_1f978cd81fb7260c8f7644039ec2c054_model.json 43KB

ordinary_textbook_1d9a847603a5e37e379738316820850d_model.json 34KB

cli_test_01.model.json 18KB

vertical_blocks.json 15KB

preproc_2_parasplit_example.json 9KB

paras_test.json 8KB

images_tables_equations.json 2KB

cla.json 811B

magic-pdf.template.json 278B

magic-pdf.json 215B

result.json 126B

magic-pdf.json 42B

cli_test_01.jsonl 6KB

cleaned_academic_literature_0b2c9c91f5232541a7ace8984df306b2.md 49KB

cleaned_academic_literature_fbdb99151e811688574c0c4c67341074.md 36KB

LICENSE.md 34KB

cleaned_academic_literature_f7904bc37cc2e25c1e3e412978854b10.md 23KB

cleaned_research_report_1f978cd81fb7260c8f7644039ec2c054.md 16KB

README_zh-CN.md 15KB

README_ja-JP.md 11KB

output_file_en_us.md 11KB

output_file_zh_cn.md 10KB

MinerU_CLA.md 7KB

README_Ubuntu_CUDA_Acceleration_en_US.md 5KB

README_Ubuntu_CUDA_Acceleration_zh_CN.md 4KB

README_Windows_CUDA_Acceleration_en_US.md 4KB

README_Windows_CUDA_Acceleration_zh_CN.md 4KB

how_to_download_models_zh_cn.md 3KB

how_to_download_models_en.md 2KB

FAQ_en_us.md 2KB

FAQ_zh_cn.md 2KB

pull_request_template.md 2KB

cleaned_ordinary_textbook_1d9a847603a5e37e379738316820850d.md 1KB

feature_request.md 1KB

academic_literature_fbdb99151e811688574c0c4c67341074.pdf 5.43MB

academic_literature_0b2c9c91f5232541a7ace8984df306b2.pdf 2.56MB

demo2.pdf 1.72MB

research_report_1f978cd81fb7260c8f7644039ec2c054.pdf 801KB

cli_test_02.pdf 565KB

cli_test_01.pdf 556KB

paper_recogPara.pdf 544KB

paper.pdf 466KB

small_ocr.pdf 443KB

demo1.pdf 329KB

ordinary_textbook_1d9a847603a5e37e379738316820850d.pdf 181KB

academic_literature_f7904bc37cc2e25c1e3e412978854b10.pdf 143KB

MinerU-logo-hq.png 1.35MB

layout_example.png 559KB

spans_example.png 550KB

project_panorama_en.png 262KB

project_panorama_zh_cn.png 247KB

MinerU-logo.png 216KB

共 292 条

Artiﬁcial Intelligence for 6G Networks: Technology

Advancement and Standardization

Muhammad K. Shehzad, Luca Rose, M. Majid Butt, István Z. Kovács, Mohamad Assaad, and Mohsen Guizani

Abstract—With the deployment of 5G networks, standards

organizations have started working on the design phase for

sixth-generation (6G) networks. 6G networks will be immensely

complex, requiring more deployment time, cost and management

efforts. On the other hand, mobile network operators demand

these networks to be intelligent, self-organizing, and cost-effective

to reduce operating expenses (OPEX). Machine learning (ML),

a branch of artiﬁcial intelligence (AI), is the answer to many

of these challenges providing pragmatic solutions, which can

entirely change the future of wireless network technologies. By

using some case study examples, we brieﬂy examine the most

compelling problems, particularly at the physical (PHY) and link

layers in cellular networks where ML can bring signiﬁcant gains.

We also review standardization activities in relation to the use

of ML in wireless networks and future timeline on readiness

of standardization bodies to adapt to these changes. Finally, we

highlight major issues in ML use in the wireless technology,

and provide potential directions to mitigate some of them in 6G

wireless networks.

Index Terms—AI, ML, Wireless networks, 3GPP, 6G.

I. INTRODUCTION

Unprecedented growth in the global cellular trafﬁc (as

shown in Fig. 1) and immense data rate demands have become

a challenge, leading wireless industry to the next-generation,

called 6G. 6G-era will bring digital, physical and biological

worlds together with the goal to improve human experience

and well-being. 6G will be operating in TeraHertz (THz)

frequencies (0.1-10 THz), hence beneﬁcial for multiple use

cases in industrial applications, providing immense data rates

(≈ 1 Tb/s), accelerating internet-of-things, and wider network

coverage. AI/ML will pave the way for THz communications

at different layers [2], e.g., supporting channel acquisition [3]

and modulation classiﬁcation [4] at PHY. Similarly, at the

link layer, beamforming design and channel allocation can

exploit ML [2]. In THz systems, a channel can signiﬁcantly

vary at a micrometer scale, resulting in a tremendous increase

in channel estimation frequency and corresponding overhead.

ML algorithms can counter this issue by using, e.g., improved

channel prediction techniques [3], [5].

Muhammad K. Shehzad (corresponding author) is with Nokia

Bell-Labs, France, and Laboratoire des Signaux et Systèmes,

CentraleSupelec, CNRS, University of Paris-Saclay, France.

({muhammad.shehzad}@nokia.com/centralesupelec.fr).

Luca Rose, and M. Majid Butt are with Nokia Bell-Labs, France.

({luca.rose, majid.butt}@nokia-bell-labs.com).

István Z. Kovács is with Nokia Standards, Aalborg, Denmark. (ist-

van.kovacs@nokia.com).

Mohamad Assaad is with Laboratoire des Signaux et Systèmes,

CentraleSupelec, CNRS, University of Paris-Saclay, France.

(mohamad.assaad@centralesupelec.fr).

Mohsen Guizani is with Mohamed Bin Zayed University of Artiﬁcial

Intelligence (MBZUAI), Abu Dhabi, UAE. (mguizani@ieee.org).

2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030

Year

100

Billion

M2M Users

MBB Users

Fig. 1. Estimation of global mobile subscriptions in machine-to-machine

(M2M) and mobile broadband (MBB) from 2020 to 2030. Source: ITU-R

Report M.2370-0 [1].

Recently, fast-growing deployment of 5G has opened up

many challenges, including massive complexity in network

architecture, low latency, high cost, power consumption, and

deployment of hybrid Long-Term Evolution (LTE) new radio

(NR), leading to difﬁculties in network optimization. In such

a complex scenario, the network intelligence has become

a major focus as it will play a pivotal role in complex

problem solving [6], e.g., self-healing, self-optimization, and

self-conﬁguration of a network [7].

Future networks will become “cognitive” in a way that

many aspects such as spectrum sensing/sharing, slicing, radio

resource management (RRM), and mobility management, will

be ML-based. Further, it is expected that ML will impact

6G air interface fundamentally and it will be designed to

support ML natively [8]. Several recent research attempts,

e.g., [9], propose different road maps for 6G, but they do not

address standardization timeline and related issues regarding

application of ML in 6G. Albeit, to some extent, [10] gives an

overview of ML and standardization; nevertheless, ML-related

technical challenges and its applications from an industrial and

standardization perspective are not addressed.

Reconﬁgurable intelligent surface (RIS) and non-orthogonal

multiple access (NOMA) are two key technologies for 6G [11].

RIS can re-engineer electromagnetic waves, hence beneﬁcial to

deliver the information where obstacles block the destination.

RIS can be integrated with ML, allowing RIS to acquire envi-

arXiv:2204.00914v1 [eess.SP] 2 Apr 2022

Supervised Learning

Task Driven

(Predict next

value)

Reinforcement Learning

Learning from

Mistakes

Regression/

Classification

Deep Learning

• ANNs

• RNNs

• CNNS

• SVMs

• NNs

• Bayesian

Networks

• K-means

• PCA

• Mixture

Models

• Q-Learning

• Fuzzy Q-

Learning

• Deep Q-

Learning

Channel

Prediction

Positioning

Slicing Handover

Spectrum

Sensing

Error

Correction

Wi-Fi

Fingerprinting

Channel

Estimation

Drone’s

Placement

Beamfor

ming

Localization

Channel

Modeling

Dynamic

Spectrum

Access

Caching

Power

Control

RRM

Unsupervised Learning

Data Driven

(Clustering)

Machine Learning

Clustering/

Dimensionality Reduction

Policy Evaluation

Major Applications in Wireless Networks

Major Tools

Fig. 2. An overview of ML paradigms, major tools, and applications in wireless networks.

ronmental information by conﬁguring various sensors, while

ML can learn dynamic parameters intelligently, reducing the

computation cost of RIS-based networks. Similarly, NOMA

is a promising access technique for 6G. In ML-empowered

NOMA-based networks, gNodeBs (gNB) can intelligently de-

ﬁne their control policy and improve decision-making ability.

Today’s networks use model-based methods to optimize

various network functions providing characteristics of the

process involved. However, these models might be too com-

plex to be implemented in a realistic time frame or they

include a great level of abstraction to function in a general

environment. In contrast, ML-based solutions can adapt to

real-time (RT) scenario changes and localized characteristics,

learning the speciﬁc environment around the transceivers. The

contributions of this article are twofold:

• We look at the above-mentioned problems from an in-

dustrial perspective and outline the gap between research

and practice.

• We review standardization activities in the context of

adopting ML in various aspects of wireless communica-

tions, e.g., channel acquisition, positioning. Furthermore,

we highlight major issues and possible research directions

in relation to the use of ML in wireless networks.

II. OVERVIEW OF ML TECHNIQUES IN WIRELESS

NETWORKS

ML is a process of training machines through data without

explicit programming. Broadly speaking, ML consists of three

paradigms: unsupervised learning, supervised learning, and

reinforcement learning (RL). All these paradigms have a

training/exploration phase to optimize a learning algorithm

that later can be used in prediction/exploitation phase to infer

on unknown inputs. As shown in Fig. 2, we brieﬂy summarize

them by providing some use cases in wireless networks.

1) Supervised Learning: Supervised learning exploits a

labelled data set to learn a (hidden) function that maps an input

to an expected output based on the examples. The standard

techniques used to solve supervised learning-based problems

are artiﬁcial neural networks (ANNs), support vector ma-

chines (SVMs), Bayesian networks, recurrent neural networks

(RNNs), and convolutional neural networks (CNNs).

2) Unsupervised Learning: Unsupervised learning does not

learn from labelled data, instead, training is based on an

unlabelled data set. K-means and principal component analysis

(PCA) are examples of two major tools used for clustering and

dimensionality reduction, respectively.

3) Reinforcement Learning: RL is not based on training

but rather the agent/decision-maker learns and decides online,

maximizing a long-term reward. RL is beneﬁcial in control

problems where the agent adapts to changing environmental

conditions, e.g., uplink power control.

Motivated by the considerable beneﬁts of ML in various

ﬁelds, its applications have also been considered in wireless

networks almost at all layers of communication. Here, we fo-

cus on its impact on radio access networks (RAN), particularly

PHY and link layers. Based on ML tools, given in Fig. 2, some

case studies will be explained later in Section III.

A. Machine Learning at PHY

At PHY, many optimization problems are non-convex, e.g.,

sum-rate maximization. ML is a powerful tool to ﬁnd good

solution(s) for such non-convex optimization problems. Based

on advanced learning algorithms, 6G networks provide the

following major advantages by using ML.

• ML can be effective to deal with network complexity. 6G

networks will be more complex due to numerous network

topologies, immense growth in the cellular users, stag-

gering data rate demands, complex air interface, vast net-

work coordination methods, etc. Forecasting considerable

complexity of 6G networks, the derivation of optimum

performance solutions is nearly infeasible without ML.

• ML can play a vital role to deal with model deﬁcit

problems. Current cellular networks are amenable for

mathematical derivation, for instance, information theory

gives closed-form expressions for various problems such

as Shannon theorem. However, the inherent complexity

of 6G networks hinders the possibility of exploiting

closed-form analytical expression(s), which can be due,

for instance, to non-linearities either in the channel or

network devices. ML offers an efﬁcient way to deal

with non-linearities, providing feasible solution(s) in a

tractable manner.

• ML can cope with algorithm deﬁcit problems. In current

cellular networks, many optimal algorithms, although

well-characterized, are impractical to be implemented.

Considering the example of multiple-input multiple-

output (MIMO) systems where optimal solutions are

known (e.g., dirty paper coding), they are overlooked in

favour of linear solutions, e.g., linear minimum mean-

squared error. It is envisaged that ML can pave the way

to implement more efﬁcient yet practical solutions.

ML has been used to study various PHY issues, and without

being exhaustive, some of the recent areas include:

• CNNs are used for modulation classiﬁcation in [4].

• An RNN-based wireless channel predictor [5] is used in

[3], explained in Section III-C, to deal with inaccurate

channel state information (CSI).

III. WIRELESS NETWORKS: CASE STUDIES

In this section, we present three use cases to demonstrate

the use of ML techniques in industrial wireless networks. ML

tools utilized for these use cases are depicted in Fig. 2.

A. UE Positioning

Highly accurate user equipment (UE) positioning is one

of the prime considerations for Third Generation Partnership

Project (3GPP) studies beyond Release 15. Various angle and

time-of-arrival-based methods are used to determine UE po-

sitioning in today’s cellular networks. All of these methods

require triangulation techniques to resolve UE position and

suffer from time synchronization errors.

We studied UE position by using radio frequency (RF)

ﬁngerprinting and two ML techniques, namely deep learning

and decision tree, for an outdoor scenario [12]. Serving cell

Reference Signal Received Power (RSRP) as well as neighbor

cell RSRP values were used as features to train a deep neural

network (DNN). As shown in Fig. 3, nearly 5 m accuracy is

achieved for DNN when only 4 serving cell RSRP values and

corresponding beam IDs are considered as a feature input,

while it improves to nearly 1 m when 2 more RSRP values

from the strongest neighboring cells, respective cell and beam

IDs are added to the input feature set. The decision tree, a

less complex algorithm as compared to DNN, provides about

2 m accuracy using data from both serving and neighboring

cell beams as an input feature. The mean accuracy of nearly

1 m obtained from DNN is comparable to the accuracy level

achieved with traditional methods without requiring triangu-

lation and does not suffer from signal timing synchronization

issues.

B. ML-Assisted Proactive Mobility

For seamless and efﬁcient mobility, a well optimized net-

work should reduce the number of Handover (HO) events

while avoiding Handover Failures (HOF) and Radio Link

Failures (RLF). An emerging approach is to utilize ML-based

评论收藏

内容反馈

版权申诉

lsx202406

粉丝: 2481
资源: 5595

基于Python语言的MinerU设计源码从GitHub导入

基于Python和Shell的GitHub导入的imagehash设计源码

基于C++和Python的HdmapEngine设计源码fork自GitHub项目

基于人脸识别的Python教室管理系统源码.zip

基于Python的购物商城管理系统源码.zip

基于java的开发源码-github-Java-api.zip

基于Python语言的AutoMihoyoBBS设计源码，支持FGO导入功能

基于Python语言的蓝鲸智云标准运维(SOPS)GitHub自动化同步设计源码

Python 如何通过爬虫实现GitHub网页的模拟登录 Python源码

基于Python的luma.core设计源码库从github镜像获取

基于Python的Flask基础项目设计源码

Python-魔改Pandora源码，GitHUB上的Pandora

基于python实现一个简单的会议室预约系统源码(课程作业).zip

基于GitHub的loonflow_ro项目Python同步设计源码

Python基于Django框架实现的一个学生信息管理系统源码.zip

基于Python和Java语言的汉语言处理工具HanLP设计源码

基于Python开发的MillionHeroAssistant设计源码下载

毕业设计、JAVA毕业设计、C#毕业设计；github 项目源码收集.zip

基于C++的pybind11跨语言Python绑定设计源码

基于Go语言的rook项目设计源码学习与导入指南

一个纯粹的Python库存管理系统源码.zip

基于Python和GitHub同步的akshare设计源码下载工具

基于C++的pybind11跨语言Python设计源码

Python-基于Selenium爬虫的自动每天点绿github主页的python脚本

python-github-projects, 在Github上，收集并分类 python 项目.zip

基于Python的Appium自动化测试框架搬运GitHub项目设计源码

基于Python的PyQt5图形界面设计源码分析

Python-HelloGitHubcom网站源码

基于GitHub平台的Python及Shell语言兼容的公共API设计源码库

基于Python和Shell语言的HASS与小米MIoT设计源码

最新资源