没有合适的资源？快使用搜索试试~ 我知道了~

文库首页人工智能深度学习Notes on DNNs

Notes on DNNs

dnn

python

model

需积分: 5 0 下载量 116 浏览量 2024-02-19 20:53:36 上传评论收藏 1.9MB PDF 举报

温馨提示

试读

29页

This is a thorough guide to tell you ways to boost the performance of your DNN! This is a thorough guide to tell you ways to boost the performance of your DNN! This is a thorough guide to tell you ways to boost the performance of your DNN! This is a thorough guide to tell you ways to boost the performance of your DNN! This is a thorough guide to tell you ways to boost the performance of your DNN! This is a thorough guide to tell you ways to boost the performance of your DNN! DNN! DNN! DNN! DNN!

资源推荐

资源详情

资源评论

SOUND SOURCE LOCALIZATION BASED ON DEEP NEURAL NETWORKS WITH DIRECTIONAL ACTIVATE FUNCTION EXPLOITING PHASE INFORMATION

This paper describes sound source localization (SSL) based on deep neural networks (DNNs) using discriminative training. A na¨ıve DNNs for SSL can be configured as follows. Input is the frequency-...

Compressed-DNNs-Forget:深度神经网络压缩实验

Compressed-DNNs-Forget:深度神经网络压缩实验

Hands-On Transfer Learning with Python

The book starts with the key essential concepts of ML and DL, followed by depiction and coverage of important DL architectures such as convolutional neural networks (CNNs), deep neural networks (DNNs)

CNNTricks_slide_Must Know Tricks in DNNs（魏秀参）

CNNTricks_slide_Must Know Tricks in DNNs（魏秀参）

自适应精度训练（AdaPT）DNNs的动态定点量化训练方法_Adaptive Precision Training (AdaPT

自适应精度训练（AdaPT）DNNs的动态定点量化训练方法_Adaptive Precision Training (AdaPT) A dynamic fixed point quantized training approach for DNNs.pdf

Dynamicneuralnetworks_Neuralnetworks_soninlaw_DNNS_非线性模型_非线性控制_源

动态神经网络(DNNs)具有重要的特性，使其易于与基于状态空间模型和微分几何的非线性控制方法一起使用，例如反馈线性化。

Python-EfficientDNNs深度网络压缩加速最新进展列表

Collection of recent methods on DNN compression and acceleration

藏经阁-When DNNs go wrong – adversarial examples and what we can le

藏经阁-When DNNs go wrong – adversarial examples and what we can le

Hands On Transfer Learning with Py Implement Advanced DL and NN Models Using T,K

dory:在基于PULP的SoC上部署深度神经网络的工具

DORY：面向记忆的部署 DORY是一种自动工具，可在通常少于1MB的片上SRAM存储器的低成本MCU上部署DNN。参考如果您使用DORY工具部署... title={DORY: Automatic End-to-End Deployment of Real-World DNNs on Low-Cost

DeepPose-Human Pose Estimation via Deep Neural Networks

on Deep Neural Networks (DNNs). The pose estimation is formulated as a DNN-based regression problem towards body joints. We present a cascade of such DNN regressors which results in high precision ...

Deep learning for time series classification a review.pdf

few have considered Deep Neural Networks (DNNs) to perform this task. This is surprising as deep learning has seen very successful applications in the last years. DNNs have indeed revolutionized the

Tutorial-on-DNN-All-Parts.pdf

DNNs • Be able to evaluate different implementations of DNN with benchmarks and comparison metrics • Understand the tradeoffs between various architectures and platforms • Assess the utility ...

Deep Neural Networks in a Mathematical Framework-Springer(2018).pdf

On the other hand, the framework that we will develop strictly operates over these vector spaces, affording a more natural mathematical description of DNNs once the objects that we use are well ...

阿里云DDNS客户端

5星 · 资源好评率100%

支持阿里云DDNS解析服务，运行该客户端即可实现ADSL拨号动态DDNS解析，满足普通用户ADSL IP不断变化的DNS解析需求，账号和密码采用AES加密存储和加密显示

【8】Deep neural networks

new methods have been shown to outperform GMMs on a variety of speech recognition benchmarks, sometimes by a large margin. This article provides an overview of this progress and represents the shared

基于Python+pytorch的图像处理+附完整代码图像处理，能够轻松实现图像的读取、显示、裁剪等还有机器学习等操作

Python+PyTorch：图像处理界的“瑞士军刀” 在图像处理这个充满魔法的世界里，Python和PyTorch这对黄金搭档，就像一位技艺高超的魔法师和一把无所不能的“瑞士军刀”，总能轻松解决各种看似棘手的难题。它们以高效、灵活和强大的特性，引领着图像处理技术的发展潮流，让无数开发者为之倾倒。Python，这位优雅的魔法师，以其简洁易懂的语法和丰富的库资源，赢得了广大开发者喜爱。无论是数据处理

python大作业含爬虫、数据可视化、地图、报告、及源码（2016-2021全国各地区粮食产量）.rar

5星 · 资源好评率100%

（含源码及报告）本程序分析了自2016年到2021年（外加）每年我国原油加工的产量，并且分析了2020年全国各地区原油加工量等，含饼状图，柱状图，折线图，数据在地图上显示。运行本程序需要requests、bs4、csv、pandas、matplotlib、pyecharts库的支持，如果缺少某库请自行安装后再运行。文件含6个excel表，若干个csv文件以及一个名字为render的html文件（需

《点燃我温暖你》中李峋的同款爱心代码

5星 · 资源好评率100%

python做的《点燃我温暖你》中李峋的同款爱心代码，最还原的

Python金融量化的高级库：TA-Lib-0.4.24（包含python3.7、3.8、3.9、3.10的32位和64位版本）

TA-Lib（Technical Analysis Library, 即技术分析库）是Python金融量化的高级库，涵盖了150多种股票、期货交易软件中常用的技术分析指标，如MACD、RSI、KDJ、动量指标、布林带等。但很多人安装指标计算ta-lib库就总报错，就可以在这里找到包下载后安装。文件举例：TA_Lib‑0.4.24‑cp37‑cp37m‑win_amd64.whl 命名解释：包名

第十五届蓝桥杯大赛软件赛省赛-PythonB组题目

您正在寻找的是第十五届蓝桥杯大赛软件赛省赛Python B组的题目全集。蓝桥杯大赛作为国内知名的计算机程序设计竞赛，一直以来都以其高水平的题目和严格的评选标准而备受瞩目。本次大赛的Python B组题目更是涵盖了众多编程领域的知识点，无论是算法设计、数据结构还是编程技巧，都考验了参赛者的深厚实力。这份题目全集以PDF格式呈现，清晰易读，方便您随时查阅和学习。每一道题目都经过精心设计和筛选，旨在

大麦网抢票脚本【Python脚本】

Python脚本，使用Selenium 模拟浏览器操作。在使用 Chrome 浏览器，用户可以使用鼠标滑动、按键点击以及键盘输入，作为信号输入设备向浏览器传达指令，浏览器收到指令后执行渲染。这里提到的 Selenium WebDriver 是对浏览器提供的原生 API 进行封装，使用这套 API 可以操控浏览器的开启、关闭，打开网页，操作界面元素，控制 Cookie。简单说就是，可以通过写代码

Python数据分析项目实践，包括数据读取、评估、清洗、分析、可视化机器学习相关内容等

Python数据分析项目实践，包括数据读取、评估、清洗、分析、可视化机器学习相关内容等

人脸识别系统OpenCV+dlib+python（含数据库）Pyqt5界面设计项目源码毕业设计

5星 · 资源好评率100%

一、项目主要技术 Python语言、dlib、OpenCV、Pyqt5界面设计、sqlite3数据库本系统使用dlib作为人脸识别工具，dlib提供一个方法可将人脸图片数据映射到128维度的空间向量，如果两张图片来源于同一个人，那么两个图片所映射的空间向量距离就很近，否则就会很远。因此，可以通过提取图片并映射到128维空间向量再度量它们的欧氏距离是否足够小来判定是否为同一个人。二

YOLOv8-火焰识别（火焰数据集+代码+GUI界面+内置训练好的模型文件）

教程请参考：https://blog.csdn.net/Little_Carter/article/details/131387425 欢迎浏览我的最新资源，这个全面的资源是为了帮助研究人员和开发者在火灾预防和安全监控领域取得突破而设计的。本资源包含以下几个关键部分： 1、火焰数据集：精心策划和注释的高质量火焰图像集，覆盖了不同类型和大小的火焰场景。这个数据集对于训练和测试火焰检测算法至关重要。

Python学习笔记(干货) 中文PDF完整版.pdf

In[]:

DNN

mission: get the best DNN

model: DNN

Ways to solve Gradient Boosting or Gradient vanishing problem

during the value passing through layers

Better Initialization - Speeding up training Meanwhile

Xavier/Glorot Initialization - when using sigmoid function as an activation function

use normal distribution with mean 0 and variance

where

or use uniform distribution between -r and r, with

Other Useful Initializers

usually, use LeCun with a normal distribution

Default: Glorot initialization with a uniform distribution

𝑓𝑎

𝑛

𝑎𝑣𝑔

𝑓𝑎

𝑛

𝑎𝑣𝑔

𝑓𝑎

𝑛

𝑖𝑛

𝑛

𝑜𝑢𝑡

𝑓𝑎

is the number of inputs of a layer, similarly with

𝑓𝑎

𝑛

𝑖𝑛

𝑛

𝑜𝑢𝑡

𝑟

3σ

⎯ ⎯⎯⎯⎯

√

'''from google.colab import drive

drive.mount('/content/drive')'''

In[19]:

In[]:

Better Activation Function - Speeding up training Meanwhile

Though ReLU works well, it may lead to a so-called problem dying ReLUs. To solve it, we use the variants of it.

Luckily, we have a variant of ReLU.

Prevent Dying ReLUs

They can be used in tf.keras.layers.LeakyReLU or tf.keras.layers.PReLU

Leaky ReLU - He Initialization

LeakyReLU_α(z)=max(αz,z) where α is the hyperparameter and α<1

α=0.3 by default

In[]:

RReLU - He Initialization

randomized Leaky ReLU - randomly set α to train and use the mean of α to test

PReLU is the Best for Large Image Set

1. Tend to overfit small sets.

2. Use He initialization.

import

tensorflow

import

numpy

import

math

dense

tf.keras.layers.Dense(50, activation

'relu', kernel_initializer

'he_normal')

# you can set

# now change fan_{in} into fan_{avg}

he_uni_init

tf.keras.initializers.VarianceScaling(scale

2., mode

'fan_avg', distribution

'uniform'

dense

tf.keras.layers.Dense(50, activation

'sigmoid', kernel_initializer

he_uni_init)

dense

tf.keras.layers.Dense(50, activation

tf.keras.layers.LeakyReLU(alpha

0.2), kernel_initialize

# this is equivalent when creating model

'''

tf.keras.layers.Dense(50, kernel_initializer='he_normal'),

tf.keras.layers.LeakyReLU(alpha=0.2),

'''

In[]:

Smooth ReLUs

A not smooth gradient will let the steps bounce around the optimum. Here comes Smooth ReLU.

ELU

activation='elu'

alpha=1 by default

use He initialization

a little slower during the whole test process

SELU

activation='selu'

self-normalization conditions

1. standardized inputs

2. use LeCun to initialize hidden layers - kernel_initializer='lecun_normal'

3. use ELU in other architectures apart from MLP

4. no regularization

Other Activation Functions that Outperforms SELU

GELU

where Φ(z) is CDF

𝐺𝐸𝐿𝑈

(

𝑧

) =

𝑧

Φ(

𝑧

)

dense

tf.keras.layers.Dense(50, activation

tf.keras.layers.PReLU(), kernel_initializer

'he_normal'

# this is equivalent when creating model

'''

tf.keras.layers.Dense(50, kernel_initializer='he_normal'),

tf.keras.layers.PReLU(),

'''

outperforms the activation functions metioned before, but it is computationally intensive

=zσ(1.702z) or so, similar performance, but less time

Swish

β is the hyperparameter, or trainable parameter

Remark: β=1 - set activation='gelu'

Mish

Mish(z)=ztanh(softplus(z))， where softplus(z)=log(1+exp(z))

𝑆𝑤𝑖𝑠

(

𝑧

) =

𝑧

σ(β

𝑧

)

ℎ

How to choose activation function?

1. simple tasks - ReLU

2. complex tasks without caring more time - Swish with learnable β

3. running time matters and task is complicated - Leaky ReLU (with learnable α)

4. deep MLP - try SELU - make sure conditions are satisfied ...

Use cross-validation to try other activation function if permitted.

Stabilize gradients

Batch Normalization - Speeding up training Meanwhile

Steps:

1. per layer - compute the mean and the modified standard deviation of each small batch

2. scale and shift each normalized inputs - these parameters can be learnt

3. those outputs are the transformed value

where ε is the smoothing term.

Add this operation just before or after the activation function of each hidden layer.

In the case where we have instances rather than batches, or the dataset is too small or not i.i.d., we can

1. wait until the whole data is updated and then calculate the whole means and standard deviations

2. (recommended) using the moving averages and standard deviations (and this is what Keras does when we

use BatchNormalization class)

In consideration of robustness, we usually compute the means and standard deviations by using exponetial

moving average which is implemented batch by batch.

where momentum is usually set close to 1, i.e., often set to 0.9 or 0.99

= momentum × + (1

−

momentum) ×

𝜇

running

𝜇

running

𝜇

batch

= momentum × + (1

−

momentum) ×

𝜎

running

𝜎

running

𝜎

batch

In[]:

μ and σ are learnt by the algorithm above (batch by batch), whereas γ and β are learnt by SGD with

backpropagation.

It wins the great performance of ImageNet and DNN.

BN gives more space for activation functions and weight initialization, solving the vanishing or exploding

gradient problems enormously.

BNlayer

tf.keras.layers.BatchNormalization()

剩余28页未读，继续阅读

评论收藏

内容反馈

资源评论

资源反馈

评论星级较低，若资源使用遇到问题可联系上传者，3个工作日内问题未解决可申请退款~

AI是这个时代的魔法

粉丝: 22
资源: 15

上传资源快速赚钱

我的内容管理展开

我的资源快来上传第一个资源

我的收益

登录查看自己的收益

我的积分登录查看自己的积分

我的C币登录后查看C币余额

我的收藏

我的下载

下载帮助

前往需求广场，查看用户热搜

Notes on DNNs

Dynamic Network Surgery for Efficient DNNs的makefile文件

Dynamic Network Surgery for Efficient DNNs的makefile.config文件

Normalization Methods for Training DNNs

Python库 | dnns-1.4.4.tar.gz

SOUND SOURCE LOCALIZATION BASED ON DEEP NEURAL NETWORKS WITH DIRECTIONAL ACTIVATE FUNCTION EXPLOITING PHASE INFORMATION

Compressed-DNNs-Forget:深度神经网络压缩实验

Hands-On Transfer Learning with Python

CNNTricks_slide_Must Know Tricks in DNNs（魏秀参）

自适应精度训练（AdaPT）DNNs的动态定点量化训练方法_Adaptive Precision Training (AdaPT

Dynamicneuralnetworks_Neuralnetworks_soninlaw_DNNS_非线性模型_非线性控制_源

Python-EfficientDNNs深度网络压缩加速最新进展列表

藏经阁-When DNNs go wrong – adversarial examples and what we can le

Hands On Transfer Learning with Py Implement Advanced DL and NN Models Using T,K

dory:在基于PULP的SoC上部署深度神经网络的工具

DeepPose-Human Pose Estimation via Deep Neural Networks

Deep learning for time series classification a review.pdf

Tutorial-on-DNN-All-Parts.pdf

Deep Neural Networks in a Mathematical Framework-Springer(2018).pdf

阿里云DDNS客户端

【8】Deep neural networks

基于Python+pytorch的图像处理+附完整代码图像处理，能够轻松实现图像的读取、显示、裁剪等还有机器学习等操作

python大作业 含爬虫、数据可视化、地图、报告、及源码（2016-2021全国各地区粮食产量）.rar

《点燃我温暖你》中李峋的同款爱心代码

Python金融量化的高级库：TA-Lib-0.4.24（包含python3.7、3.8、3.9、3.10的32位和64位版本）

第十五届蓝桥杯大赛软件赛省赛-PythonB组题目

大麦网抢票脚本【Python脚本】

Python数据分析项目实践，包括数据读取、评估、清洗、分析、可视化机器学习相关内容等

人脸识别系统OpenCV+dlib+python（含数据库）Pyqt5界面设计 项目源码 毕业设计

YOLOv8-火焰识别（火焰数据集+代码+GUI界面+内置训练好的模型文件）

Python学习笔记(干货) 中文PDF完整版.pdf

最新资源

python大作业含爬虫、数据可视化、地图、报告、及源码（2016-2021全国各地区粮食产量）.rar

人脸识别系统OpenCV+dlib+python（含数据库）Pyqt5界面设计项目源码毕业设计