You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: gsoc.md
+49-82
Original file line number
Diff line number
Diff line change
@@ -6,112 +6,79 @@ title = "Google Summer of Code"
6
6
<h1>FluxML Projects - Summer of Code</h1>
7
7
~~~
8
8
9
-
Flux usually takes part in [Google Summer of Code](https://summerofcode.withgoogle.com) as a NumFocus organization. We follow the same [rules and application guidelines](https://julialang.org/jsoc/projects/) as Julia, so please check there for more information on applying. Below are a set of ideas for potential projects (though you are welcome to explore anything you are interested in).
9
+
Flux usually takes part in [Google Summer of Code](https://summerofcode.withgoogle.com) as a NumFocus organization. We follow the same [rules and application guidelines](https://julialang.org/jsoc/projects/) as Julia, so please check there for more information on applying. Below are a set of ideas for potential projects (though you are welcome to explore anything you are interested in).**Please note that year for the idea list. Project ideas from a previous year will not always carry over to a new year.**
10
10
11
-
Flux projects are typically very competitive; we encourage you to get started early, as successful contributors typically have early PRs or working prototypes as part of the application. It is a good idea to simply start contributing via issue discussion and PRs and let a project grow from there; you can take a look at [this list of issues](https://github.com/FluxML/Flux.jl/issues?q=is%3Aopen+is%3Aissue+label%3A%22help+wanted%22) for some starter contributions.
11
+
Flux projects are typically very competitive; we encourage you to get started early, as successful contributors typically have early PRs or working prototypes as part of the application. It is a good idea to simply start contributing via issue discussion and PRs and let a project grow from there; you can take a look at [this list of issues](https://github.com/FluxML/Flux.jl/issues?q=is%3Aopen+is%3Aissue+label%3A%22help+wanted%22) for some starter contributions. Please see the [contributing guide](https://github.com/FluxML/Flux.jl/blob/master/CONTRIBUTING.md) for help first.
12
12
13
-
## Metalhead.jl Developement
13
+
#FluxML GSoC 2023 Ideas List
14
14
15
-
**Difficulty:** Medium (175h)
15
+
## Writing Julia-native kernels for common NN operations
16
16
17
-
**Expected outcomes:** Help us improve [Metalhead.jl](https://github.com/FluxML/Metalhead.jl) by
18
-
@@tight-list
19
-
- adding new models
20
-
- porting pre-trained weights
21
-
- extending the model interfaces to make them more customizable
22
-
@@
17
+
Implement optimized kernels for common neural network operations for which we don't already have Julia-native implementations. This project will require experience with GPU kernel writing and performance optimizations.
23
18
24
-
**Skills:**Familiarity with vision model architectures and Flux.jl
Many ML frameworks are making the move away from vendor-specific libraries (like CUBLAS, CUDNN, etc.) towards more generic, JIT-compiled implementations of ML-related kernels, like BLAS, softmax, ReLU, etc. The reasons for this move are many-fold:
24
+
- Vendor-provided libraries often only work on that vendor's hardware and software
25
+
- These libraries only support certain element types, tensor shapes/sizes, and limited array view/stride/transpose support
26
+
- These libraries often expect to be executed from the host, without a device-side launchable equivalent
27
+
- These libraries have unreliable build systems or are binary blobs
29
28
30
-
**Difficulty:** Medium (350h)
29
+
Improving this state of affairs for Flux will involve using Julia's existing GPU and compute kernel libraries (e.g KernelAbstractions.jl) to implement various accelerated, cross-vendor routines. These kernels should be both composable and performance competitive with Flux's current generic code paths. Examples of routines specifically useful for implementing Neural Networks include:
31
30
32
-
In this project, you will assist the [ML community team](https://julialang.zulipchat.com/#narrow/stream/237432-ml-ecosystem-coordination) with building time series methods for FastAI.jl on top of the existing JuliaML + FluxML ecosystem packages. Some familiarity with the following Julia packages is preferred, but it is not required:
The ideal candidate should have experience with what operations are used in popular ML models and how they are commonly implemented on GPU. This includes experience writing and benchmarking high performance GPU kernels. Because kernels will be required for both training and inference, an understanding of automatic differentiation (AD) is also highly recommended.
39
38
40
-
**Expected outcomes:** You will
41
-
@@tight-list
42
-
- load a working time series dataset using the FastAI.jl data registry
43
-
- create new block methods for time series tasks
44
-
- load at least one working time series model into a learner
45
-
- develop an example tutorial that ties all the previous steps together
- Experience with low-level GPU kernel programming is strongly recommended.
45
+
- Experience with common primitive machine learning ops (forwards and backwards passes) and their interaction is recommended.
46
+
- Familiarity with existing prior art such as [tiny-cuda-nn](https://github.com/NVlabs/tiny-cuda-nn) is preferred.
51
47
52
-
##FastAI.jl Text Development
48
+
### Your contributions
53
49
54
-
**Difficulty:** Medium (350h)
50
+
- A new package containing the optimized kernels and any supporting code for integration into Flux/Flux's operation library [NNlib.jl](https://github.com/FluxML/NNlib.jl).
51
+
- Tests on CI and a simple benchmark harness for the new NN kernel library.
52
+
- A proof-of-concept example showing the kernels being used with kernel fusion on device (GPU).
55
53
56
-
In this project, you will assist the [ML community team](https://julialang.zulipchat.com/#narrow/stream/237432-ml-ecosystem-coordination) with building text methods for FastAI.jl on top of the existing JuliaML + FluxML ecosystem packages. Some familiarity with the following Julia packages is preferred, but it is not required:
## Benchmark tooling for common models and operations
63
56
64
-
**Expected outcomes:** You will
65
-
@@tight-list
66
-
- load a working text dataset using the FastAI.jl data registry
67
-
- create new block methods for textual tasks
68
-
- load at least one working text model into a learner
69
-
- develop an example tutorial that ties all the previous steps together
70
-
@@
57
+
Create a benchmarking tool for the FluxML ecosystem that we can invoke on demand from PRs. This project will require previous experience with training machine learning models at a "low-level" (i.e. without the use of tools like PyTorch Lightning).
71
58
72
-
**Skills:**Familiarity with deep learning pipelines, common practices, Flux.jl, and JuliaText
FluxML's machine learning stack is distributed across many packages, and each package is designed to function as independently as possible. This is done to maximize code-reuse across the Julia ML ecosystem. As a result, it is challenging for us to quantitively determine the performance impact of code changes without manual testing. The goal of this project is to develop a FluxML-specific benchmarking package. The package should allow us to install specific commits of various packages across the FluxML stack, then run a benchmarking suite. The test suite will include low-level operations like convolution or simple gradient calls, as well as complete end-to-end examples like a full forward/backward pass of Metalhead.jl models.
77
64
78
-
**Difficulty:** Hard (350h)
65
+
The ideal candidate should have experience with multiple ML task setups such as vision, autoregressive language modeling, time series forecasting, etc. Furthermore, some experience with Github Actions and continuous integration (CI) is suggested.
79
66
80
-
Create a library of utility functions that can consume Julia's imaging libraries to make them differentiable. With Zygote.jl, we have the platform to take a general purpose package and apply automatic differentiation to it.
## FermiNets: Generative Synthesis for Automating the Choice of Neural Architectures
93
-
94
-
**Difficulty:** Hard (175h)
95
-
96
-
The application of machine learning requires a practitioner to understand how to optimize a neural architecture for a given problem, or does it? Recently, techniques in automated machine learning, also known as AutoML, have dropped this requirement by allowing for good architectures to be found automatically. One such method is the [FermiNet](https://arxiv.org/abs/1809.05989), which employs generative synthesis to give a neural architecture which respects certain operational requirements.
97
-
98
-
**Expected outcomes:** The goal of this project is to implement the FermiNet in Flux to allow for automated synthesis of neural networks.
99
-
100
-
**Mentors:**[Chris Rackauckas](https://github.com/ChrisRackauckas) and [Dhairya Gandhi](https://github.com/DhairyaLGandhi/)
101
-
102
-
## Differentiable Rendering
103
-
104
-
**Difficulty:** Hard (350h+)
105
-
106
-
We have an existing package, [RayTracer.jl](https://github.com/avik-pal/RayTracer.jl), which is motivated by OpenDR and exists to do differentiable raytracing with Flux.jl and Zygote.jl.
107
-
108
-
**Expected outcomes:** You will
109
-
@@tight-list
110
-
- implement at least 2 alternative rendering models like NeRF, VolSDF, Neural Raytracing, etc.
111
-
- make improvements to RayTracer.jl to use the latest Flux libraries
112
-
- update RayTracer.jl for ChainRules.jl
113
-
@@
114
-
115
-
**Skills:** GPU programming, deep learning, familiarity with the literature, familiarity with defining custom adjoints
0 commit comments