Active Entries
- 1: "Narrow AGI" this year?
- 2: Tao on coordinate vs coordinate-free math reasoning
- 3: "Aging as a loss of goal-directedness"
- 4: New integrated mode for GPT-4 in ChatGPT+
- 5: Китайский новый год начнётся 10-го февраля
- 6: Automating the Search for Artificial Life with Foundation Models
- 7: "Anatomy of a Formal Proof"
- 8: C to safe Rust automatic translation using LLMs and dynamic analysis
- 9: GonzoML
- 10: Transformers as a Computational Model (workshop)
Style Credit
- Style: Neutral Good for Practicality by
Expand Cut Tags
No cut tags
no subject
Date: 2023-07-27 06:20 pm (UTC)https://pretalx.com/juliacon2023/talk/RTCDVR/
State of Julia
07-28, 09:00–09:45 (US/Eastern), 26-100
https://pretalx.com/juliacon2023/talk/QN3XGU/
Learning smoothly: machine learning with RobustNeuralNetworks.jl
07-28, 10:00–10:30 (US/Eastern), 26-100
This session's header image
Neural networks are typically sensitive to small input perturbations, leading to unexpected or brittle behaviour. We present RobustNeuralNetworks.jl: a Julia package for neural network models that are constructed to naturally satisfy robustness constraints. We discuss the theory behind our model parameterisation, give an overview of the package, and demonstrate its use in image classification, reinforcement learning, and nonlinear robotic control.
Modern machine learning relies heavily on rapidly training and evaluating neural networks in problems ranging from image classification to robotic control. However, most existing neural network architectures have no robustness certificates, making them sensitive to even small input perturbations and highly susceptible to poor data quality, adversarial attacks, and other forms of input disturbances. The few neural network architectures proposed in recent years that offer solutions to this brittle behaviour rely on explicitly enforcing constraints during training to “smooth” the network response. These methods are computationally expensive, making them slow and difficult to scale up to complex real-world problems.
Recently, we proposed the Recurrent Equilibrium Network (REN) architecture as a computationally efficient solution to these problems. The REN architecture is flexible in that it includes all commonly used neural network models, such as fully-connected networks, convolutional neural networks, and recurrent neural networks. The weight matrices and bias vectors in a REN are directly parameterised to naturally satisfy behavioural constraints chosen by the user. For example, the user can build a REN with a given Lipschitz constant to ensure the output of the network is quantifiably less sensitive to unexpected input perturbations. Other common options include contracting RENs and input/output passive RENs.
The direct parameterisation of RENs means that no additional constrained optimization methods are needed to train the networks to be less sensitive to attacks or perturbations. We can therefore train RENs with standard, unconstrained optimization methods (such as gradient descent) while also guaranteeing their robustness. Achieving the “best of both worlds” in this way is unique to our REN model class, and allows us to freely train RENs for common machine learning problems as well as more difficult applications where safety and robustness are critical.
In this talk, we will present our RobustNeuralNetworks.jl package. The package is built around the AbstractREN type, encoding the REN model class. It relies heavily on key features of the Julia language (such as multiple dispatch) for a neat, efficient implementation of RENs, and can be used alongside Flux.jl to solve machine learning problems with and without robustness requirements, all in native Julia.
We will give a brief introduction to the fundamental theory behind our direct parameterisation of neural networks, and outline what we mean by nonlinear robustness. We will follow this with a detailed overview of the RobustNeuralNetworks.jl package structure, including the key types and methods used to construct and implement a REN. To conclude, we will demonstrate some interesting applications of our Julia package for REN in our own research, including in:
Image classification
System identification
Learning-based control for dynamical systems
Real-time control of robotic systems via the Julia C API
Ultimately, we hope to show how RENs will be useful to the wider Julia machine learning community in both research and industry applications. For more information on the REN model class and its uses, please see our two recent papers https://arxiv.org/abs/2104.05942 and https://doi.org/10.1109/LCSYS.2022.3184847.
Nicholas Barbara
Nicholas Barbara is a PhD candidate at the Australian Centre for Robotics, within the University of Sydney. He is interested in robust machine learning, control theory, spacecraft GNC, and all things Julia.
https://pretalx.com/juliacon2023/talk/FVZXUF/ (not livestreamed/not clear if recording will be published/not sure what to do about this; I am very interested in sparseness, but it's inconvenient timing)
Sparsity: Practice-to-Theory-to-Practice
07-28, 11:00–11:25 (US/Eastern), 32-141
Join us for ASE-60, where we celebrate the life and the career of Professor Alan Stuart Edelman, on the occasion of his 60th birthday: https://math.mit.edu/events/ase60celebration/
As we all know, the entire world of computation is mostly matrix multiplies. Within this universe we do allow some variation. Specifically, all the world is mostly either dense matrix multiplies or sparse matrix multiplies. Sparse matrices are often used as a trick to solve larger problems by only storing non-zero values. As a result, there is large toolkit of powerful sparse matrix software. The availability of sparse matrix tools inspires representing a wide range of problems as sparse matrices. Notably graphs have many wonderful sparse matrix properties and many graph algorithms can be written as matrix multiplies using a variety of semirings. This inspires developing new sparse matrix software that encompasses a wide range of semiring operations. In the context of graphs, where vertex labels are diverse, it is natural to relax strict dimension constraints and make hyper-sparse matrices a full-fledged member of the sparse matrix software world. The wide availability of hyper-sparse matrices allows addressing a wide range of problems and completely new approaches to parallel computing.
https://pretalx.com/juliacon2023/talk/9ES8NF/
Falra.jl : Distributed Computing with AI Source Code Generation
07-28, 12:20–12:30 (US/Eastern), 32-124
Falra.jl in Julia provides a straightforward approach to implementing distributed computing, equipped with an AI-assisted feature for generating source code. This addition facilitates more efficient big data transformations. Tasks such as preprocessing 16TB of IoT data can be done in 1/100 of the original time. Developers are now able to generate Julia source code more easily with the aid of AI, further aiding in distributed computing tasks.
This is a real development scenario that we encountered to preprocess 6-year, 16TB historical IoT raw datasets for data cleaning and transformation. It takes 100 days to complete processing in a single-machine environment, which is time-consuming.
So, the Falra.jl was developed to allow us to divide the data cleaning and transformation tasks that we need to perform into smaller tasks. Falra.jl then automatically distributes these tasks for distributed processing. This architecture saves a lot of computing time and development costs. Through Falra.jl, we were able to complete all IoT data transformations in 1/100 of the time.
Compared to the native Julia distributed module, the advantage of Falra.jl is that developers do not need to learn how to develop the Julia distributed programming syntax. They can just use their single-machine programs as they used to do. In addition, Falra.jl can be deployed on any network that can be called via HTTPS. There is no need to deal with TCP or other network or firewall issues.
Moreover, we've enhanced our approach by integrating AI-assisted Julia source code auto-generation. This novel feature allows developers to efficiently create Julia code using artificial intelligence. Rather than manually crafting each line of code, the AI
can generate source code based on the developer's requirements, thus accelerating the development process. It makes it feasible for developers, even those unfamiliar with Julia, to quickly produce distributed programs. This AI-driven tool not only simplifies code creation but also enables the rapid adaptation and extension of the
applications under the Falra.jl . The fusion of distributed computing and AI-assisted auto-generation of Julia source code significantly boosts productivity.
Currently, we have released the Falra.jl on Github (https://github.com/bohachu/Falra.jl) for everyone to use.
Bowen Chiu
With 33 years of experience in software programming, Bowen is the founder of CAMEO Corporation. He specializes in artificial intelligence and distributed computing, with a
particular focus on the environmental sector, the educational sector, and start-ups.