awesome-deeplearning-resources

Deep learning

• Accelerating CNN inference on FPGAs: A Survey. arxiv
• Adaptive Neural Trees. arxiv
• Adding One Neuron Can Eliminate All Bad Local Minima. arxiv
• A Dual Approach to Scalable Verification of Deep Networks. arxiv
• A Geometric Theory of Higher-Order Automatic Differentiation. arxiv
• A graph-embedded deep feedforward network for disease outcome classification and feature selection using gene expression data. arxiv code
• An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution. url :star:
• A Survey on Neural Network-Based Summarization Methods. arxiv
• A Tutorial on Network Embeddings. arxiv
• A Unified Probabilistic Model for Learning Latent Factors and Their Connectivities from High-Dimensional Data. arxiv
• Backdrop: Stochastic Backpropagation. arxiv code
• Batch Kalman Normalization: Towards Training Deep Neural Networks with Micro-Batches. arxiv :star:
• Bayesian Convolutional Neural Networks. arxiv code
• Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification. arxiv
• Bayesian Layers: A Module for Neural Network Uncertainty. arxiv
• Bayesian Neural Networks. arxiv code
• Bayesian Optimization in AlphaGo. arxiv
• Beyond Word Importance: Contextual Decomposition to Extract Interactions from LSTMs. arxiv
• BindsNET: A machine learning-oriented spiking neural networks library in Python. arxiv code
• Capturing Structure Implicitly from Time-Series having Limited Data. arxiv code
• Class label autoencoder for zero-shot learning. arxiv
• Closing the AI Knowledge Gap. arxiv
• Clustering with Deep Learning: Taxonomy and New Methods. arxiv code
• Collaborative Multi-modal deep learning for the personalized product retrieval in Facebook Marketplace. arxiv
• Conditional Neural Processes. arxiv
• Convolutional Neural Networks with Recurrent Neural Filters. arxiv code
• Data Dropout: Optimizing Training Data for Convolutional Neural Networks. arxiv
• Decorrelated Batch Normalization. arxiv code
• Decoupled Networks. arxiv code
• Deep Convolutional Networks as shallow Gaussian Processes. arxiv code
• Deep Embedding Kernel. arxiv
• Deep Graph Infomax. arxiv
• Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations. arxiv code
• Deep k-Means: Re-Training and Parameter Sharing with Harder Cluster Assignments for Compressing Deep Convolutions. arxiv code
• Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning. arxiv
• Deep Learning. arxiv
• Deep Learning using Rectified Linear Units (ReLU). arxiv code
• Deep Learning with the Random Neural Network and its Applications. arxiv
• Deep Multimodal Subspace Clustering Networks. arxiv
• Deep Neural Decision Trees. arxiv code
• Deep Self-Organization: Interpretable Discrete Representation Learning on Time Series. arxiv
• Deep Super Learner: A Deep Ensemble for Classification Problems. arxiv code
• Detail-Preserving Pooling in Deep Networks. arxiv
• Detecting Dead Weights and Units in Neural Networks. arxiv
• Digging Into Self-Supervised Monocular Depth Estimation. arxiv
• Don’t Use Large Mini-Batches, Use Local SGD. arxiv
• Dropout is a special case of the stochastic delta rule: faster and more accurate deep learning. arxiv
• DroNet: Learning to Fly by Driving. pdf code
• Dynamic Graph Neural Networks. arxiv
• Dynamic Planning Networks. arxiv
• Efficient Interactive Annotation of Segmentation Datasets with Polygon-RNN++. arxiv code
• Efficient Neural Architecture Search ia Parameter Sharing. arxiv pytorch tensorflow :star:
• Entropy and mutual information in models of deep neural networks. arxiv
• EcoRNN: Fused LSTM RNN Implementation with Data Layout Optimization. arxiv
• E-swish: Adjusting Activations to Different Network Depths. arxiv
• Etymo: A New Discovery Engine for AI Research. arxiv
• Evaluating Feature Importance Estimates. arxiv
• Extremely Fast Decision Tree. arxiv pytorch
• Eyeriss v2: A Flexible and High-Performance Accelerator for Emerging Deep Neural Networks. arxiv
• Fast Decoding in Sequence Models using Discrete Latent Variables. arxiv
• FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling. arxiv code
• Fast Neural Architecture Search of Compact Semantic Segmentation Models via Auxiliary Cells. arxiv
• Foundations of Sequence-to-Sequence Modeling for Time Series. arxiv
• From Nodes to Networks: Evolving Recurrent Neural Networks. arxiv
• Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network. arxiv :star:
• Gaussian Process Behaviour in Wide Deep Neural Networks. arxiv code
• Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels. arxiv
• Geometric Understanding of Deep Learning. arxiv
• GossipGraD: Scalable Deep Learning using Gossip Communication based Asynchronous Gradient Descent. arxiv
• Gradient Acceleration in Activation Functions. arxiv
• Graph Capsule Convolutional Neural Networks. arxiv code
• Graph Partition Neural Networks for Semi-Supervised Classification. arxiv
• GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Model. arxiv code
• Group Normalization. arxiv :star:
• Hierarchical Graph Representation Learning with Differentiable Pooling. arxiv
• High-Accuracy Low-Precision Training. arxiv
• High Dimensional Bayesian Optimization Using Dropout. arxiv
• How Does Batch Normalization Help Optimization. arxiv
• How Powerful are Graph Neural Networks? pdf
• Hybrid Decision Making: When Interpretable Models Collaborate With Black-Box Models. arxiv code
• Hybrid Gradient Boosting Trees and Neural Networks for Forecasting Operating Room Data. arxiv
• Hyperbolic Neural Networks. arxiv code
• IcoRating: A Deep-Learning System for Scam ICO Identification. arxiv
• Impacts of Dirty Data: and Experimental Evaluation. arxiv code
• Implicit Autoencoders. arxiv
• Improving Multi-Person Pose Estimation using Label Correction. arxiv
• Incremental Training of Deep Convolutional Neural Networks. arxiv
• Introducing Neuromodulation in Deep Neural Networks to Learn Adaptive Behaviours. arxiv
• Labelling as an unsupervised learning problem.arxiv
• Large Data and Zero Noise Limits of Graph-Based Semi-Supervised Learning Algorithms. arxiv
• Large-Margin Classification in Hyperbolic Space. arxiv code
• LARNN: Linear Attention Recurrent Neural Network. arxiv code
• Learning Classifiers with Fenchel-Young Losses: Generalized Entropies, Margins, and Algorithms. arxiv
• Learning Latent Representations in Neural Networks for Clustering through Pseudo Supervision and Graph-based Activity Regularization. arxiv
• Learning Longer-term Dependencies in RNNs with Auxiliary Losses. arxiv
• Learning Networks from Random Walk-Based Node Similarities. arxiv code
• Learning to generate classifiers. arxiv code
• Learning to Learn Without Labels. pdf :star:
• Learning to Make Predictions on Graphs with Autoencoders. arxiv code
• Learning to Reweight Examples for Robust Deep Learning. arxiv code
• Learning to Zoom: a Saliency-Based Sampling Layer for Neural Networks. arxiv
• Learning Unsupervised Learning Rules. arxiv tensorflow :star:
• Learning What to Remember: Long-term Episodic Memory Networks for Learning from Streaming Data. arxiv
• Linear Backprop in non-linear networks. pdf
• Links: A High-Dimensional Online Clustering Method. arxiv :star:
• LiteFlowNet: A Lightweight Convolutional Neural Network for Optical Flow Estimation. arxiv
• LSTM stack-based Neural Multi-sequence Alignment TeCHnique. arxiv
• MemGEN: Memory is All You Need. arxiv
• Measuring the Effects of Data Parallelism on Neural Network Training. arxiv
• Modeling Attention Flow on Graphs. arxiv
• Modeling Dynamics with Deep Transition-Learning Networks. arxiv
• Model Selection Techniques – An Overview. arxiv
• Multi-Layered Gradient Boosting Decision Trees. arxiv :star:
• Multivariate LSTM-FCNs for Time Series Classification. arxiv code
• Network Distance Based on Laplacian Flows on Graphs. arxiv :star:
• Neural Arithmetic Logic Units. arxiv
• Neural Architecture Optimization. arxiv code
• Neural Architecture Search: A Survey. arxiv
• Neural Nearest Neighbors Networks. arxiv code
• Neural Networks Regularization Through Representation Learning. arxiv
• Not All Samples Are Created Equal: Deep Learning with Importance Sampling. arxiv
• Online normalizer calculation for softmax. arxiv
• [Best Paper] On the Convergence of Adam and Beyond. pdf :star:
• On the Theory of Variance Reduction for Stochastic Gradient Monte Carlo. arxiv
• Parallel Grid Pooling for Data Augmentation. arxiv code
• Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding. arxiv
• Piggyback: Adapting a Single Network to Multiple Tasks by Learning to Mask Weights. arxiv code
• Pitfalls of Graph Neural Network Evaluation. arxiv
• Pooling is neither necessary nor sufficient for appropriate deformation stability in CNNs. arxiv
• Probabilistic Recurrent State-Space Models. arxiv code
• Progress & Compress: A scalable framework for continual learning. arxiv
• PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume. arxiv code
• Pyramid Stereo Matching Network. arxiv code
• Random depthwise signed convolutional neural networks. arxiv
• Random Fourier Features for Kernel Ridge Regression: Approximation Bounds and Statistical Guarantees. arxiv
• Random Warping Series: A Random Features Method for Time-Series Embedding. arxiv code
• Relational recurrent neural networks. arxiv
• Representation Learning with Contrastive Predictive Coding. arxiv
• ResNet with one-neuron hidden layers is a Universal Approximator. arxiv
• Revisiting Small Batch Training for Deep Neural Networks. arxiv
• R3-Net: A Deep Network for Multi-oriented Vehicle Detection in Aerial Images and Videos. arxiv
• Rotation Equivariance and Invariance in Convolutional Neural Networks. arxiv
• Scale-free network clustering in hyperbolic and other random graphs. arxiv
• SlimNets: An Exploration of Deep Model Compression and Acceleration. arxiv code
• Smallify: Learning Network Size while Training. arxiv
• Sparsely Connected Convolutional Networks. arxiv
• SparseMAP: Differentiable Sparse Structured Inference. arxiv :star:
• SpectralNet: Spectral Clustering using Deep Neural Networks. arxiv code
• Spiking Deep Residual Network. arxiv
• Spherical CNNs. arxiv code :star:
• Step Size Matters in Deep Learning. arxiv
• Supervised classification of Dermatological diseases by Deep neural networks. arxiv code
• Supervising Unsupervised Learning with Evolutionary Algorithm in Deep Neural Network. arxiv
• Syntax-Aware Language Modeling with Recurrent Neural Networks. arxiv
• Realistic Evaluation of Deep Semi-Supervised Learning Algorithms. arxiv code
• Testing Deep Neural Networks. arxiv code
• The Lottery Ticket Hypothesis: Training Pruned Neural Networks. arxiv
• The Matrix Calculus You Need For Deep Learning. arxiv
• Theory and Algorithms for Forecasting Time Series. arxiv
• The Singular Values of Convolutional Layers. arxiv
• The unreasonable effectiveness of the forget gate. arxiv
• Time is of the Essence: Machine Learning-based Intrusion Detection in Industrial Time Series Data. arxiv
• Time Series Segmentation through Automatic Feature Learning. arxiv
• Towards a Theoretical Understanding of Batch Normalization. arxiv
• Towards Efficient Large-Scale Graph Neural Network Computing. arxiv
• Tracking Network Dynamics: a review of distances and similarity metrics. arxiv
• Training convolutional neural networks with megapixel images. url code
• Tree-CNN: A Deep Convolutional Neural Network for Lifelong Learning. arxiv
• t-SNE-CUDA: GPU-Accelerated t-SNE and its Applications to Modern Data. arxiv code
• Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooring. arxiv
• TVM: End-to-End Optimization Stack for Deep Learning. arxiv
• UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction. arxiv code
• Understanding Convolutional Neural Network Training with Information Theory. arxiv
• Understanding Individual Decisions of CNNs via Contrastive Backpropagation. arxiv
• Understanding the Disharmony between Dropout and Batch Normalization by Variance Shift. arxiv
• Understanding the Loss Surface of Neural Networks for Binary Classification. arxiv
• Universal Deep Neural Network Compression. arxiv
• URLNet: Learning a URL Representation with Deep Learning for Malicious URL Detection. arxiv code
• Wasserstein is all you need. arxiv
• Weighted Risk Minimization & Deep Learning. arxiv
• What Do We Understand About Convolutional Networks? arxiv

Attention

• Attention-based Graph Neural Network for Semi-supervised Learning. arxiv
• Attention Solves Your TSP. arxiv code
• Automatic Instrument Segmentation in Robot-Assisted Surgery Using Deep Learning. url
• Compositional Attention Networks for Machine Reasoning. arxiv tensorflow
• Hyperbolic Attention Networks. arxiv
• Inference, Learning and Attention Mechanisms that Exploit and Preserve Sparsity in Convolutional Networks. arxiv code
• MAttNet: Modular Attention Network for Referring Expression Comprehension. arxiv
• Reinforced Self-Attention Network: a Hybrid of Hard and Soft Attention for Sequence Modeling. arxiv
• Tell Me Where to Look: Guided Attention Inference Network. arxiv CODE

Generative learning

• Adversarial Attack on Graph Structured Data. arxiv
• Adversarial Attacks Against Medical Deep Learning Systems. arxiv code
• Adversarial Classification on Social Networks. arxiv
• Adversarial Logit Pairing. arxiv :star:
• Adversarial Reprogramming of Neural Networks. arxiv
• Adversarial Spheres. arxiv
• AmbientGAN: Generative models from lossy measurements. url code
• An empirical study on evaluation metrics of generative adversarial networks. arxiv code
• Anime Style Space Exploration Using Metric Learning and Generative Adversarial Networks. arxiv
• Autoencoding topology. arxiv
• CariGANs: Unpaired Photo-to-Caricature Translation. arxiv
• CartoonGAN: Generative Adversarial Networks for Photo Cartoonization. pdf pytorch
• cGANs with Projection Discriminator. pdf code
• ClusterGAN : Latent Space Clustering in Generative Adversarial Networks. arxiv
• Compositional GAN: Learning Conditional Image Composition. arxiv code
• Counterfactuals uncover the modular structure of deep generative models. arxiv
• CR-GAN: Learning Complete Representations for Multi-view Generation. arxiv code
• Deep Generative Markov State Models. arxiv code
• Deep Learning for Imbalance Data Classification using Class Expert Generative Adversarial Network. arxiv
• eCommerceGAN : A Generative Adversarial Network for E-commerce. arxiv
• Evolving Mario Levels in the Latent Space of a Deep Convolutional Generative Adversarial Network. arxiv
• Geometry Score: A Method For Comparing Generative Adversarial Networks. arxiv code
• Generating Handwritten Chinese Characters using CycleGAN. arxiv code
• Generative Adversarial Networks using Adaptive Convolution. arxiv
• Improving GANs Using Optimal Transport. arxiv
• Inverting The Generator Of A Generative Adversarial Network (II). arxiv code
• Learning Dynamics of Linear Denoising Autoencoders. arxiv code
• Learning Inverse Mappings with Adversarial Criterion. arxiv code
• New Losses for Generative Adversarial Learning. arxiv
• On Generation of Adversarial Examples using Convex Programming. arxiv code
• On the Latent Space of Wasserstein Auto-Encoders. arxiv
• Recurrent Neural Network-Based Semantic Variational Autoencoder for Sequence-to-Sequence Learning. arxiv
• Scalable Factorized Hierarchical Variational Autoencoder Training. arxiv code
• Semi-Amortized Variational Autoencoders. arxiv code
• Semi-supervised Learning on Graphs with Generative Adversarial Nets. arxiv
• Siamese networks for generating adversarial examples. arxiv
• Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks. arxiv code
• Sylvester Normalizing Flows for Variational Inference. arxiv
• Synthesizing Audio with Generative Adversarial Networks. arxiv
• tempoGAN: A Temporally Coherent, Volumetric GAN for Super-resolution Fluid Flow. arxiv
• The relativistic discriminator: a key element missing from standard GAN. arxiv code
• Understanding and Improving Interpolation in Autoencoders via an Adversarial Regularizer. arxiv code
• Unsupervised Cipher Cracking Using Discrete GANs. arxiv tensorflow
• Variational Bayesian Monte Carlo. arxiv code

Meta Learning

• Bayesian Model-Agnostic Meta-Learning. arxiv
• Meta-Learning: A Survey. arxiv
• Meta-Learning for Semi-Supervised Few-Shot Classification. arxiv code
• Reptile: a Scalable Metalearning Algorithm. arxiv

Optimization

• Averaging Weights Leads to Wider Optima and Better Generalization. arxiv
• Computational Optimal Transport. arxiv
• Energy-entropy competition and the effectiveness of stochastic gradient descent in machine learning. arxiv
• Gradient Descent Quantizes ReLU Network Features. arxiv
• L4: Practical loss-based stepsize adaptation for deep learning. arxiv code
• Sequential Preference-Based Optimization. arxiv code
• Sever: A Robust Meta-Algorithm for Stochastic Optimization. arxiv
• Shampoo: Preconditioned Stochastic Tensor Optimization. arxiv pytorch
• Optimizing for Generalization in Machine Learning with Cross-Validation Gradients. arxiv code
• WNGrad: Learn the Learning Rate in Gradient Descent. arxiv

Transfer Learning

• 3D Convolutional Encoder-Decoder Network for Low-Dose CT via Transfer Learning from a 2D Trained Network. arxiv
• Adversarial Transfer Learning. arxiv
• Arbitrary Style Transfer with Style-Attentional Networks. arxiv
• A Survey on Deep Transfer Learning. arxiv
• Avatar-Net: Multi-scale Zero-shot Style Transfer by Feature Decoration. arxiv
• Capsule networks for low-data transfer learning. arxiv
• Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer. arxiv code
• Learn from Your Neighbor: Learning Multi-modal Mappings from Sparse Annotations. arxiv
• [Best Paper] Taskonomy: Disentangling Task Transfer Learning. arxiv :star:

Zero/One Shot Learning

• A Large-scale Attribute Dataset for Zero-shot Learning. arxiv
• Deep Triplet Ranking Networks for One-Shot Recognition. arxiv
• One-Shot Learning using Mixture of Variational Autoencoders: a Generalization Learning approach. arxiv
• One-Shot Unsupervised Cross Domain Translation. arxiv code
• Preserving Semantic Relations for Zero-Shot Learning. arxiv

This site is open source. Improve this page.