[RFC]: add BLAS bindings and implementations for linear algebra

[ShabiShett07]

Full name

Shabareesh Shetty

University status

Yes

University name

NMAMIT Nitte

University program

Bachelor's of Technology in Robotics and Artificial Intelligence

Expected graduation

2027

Short biography

I am a 2nd year undergraduate student at NMAMIT Nitte, currently pursuing my Bachelor's of Technology in the field of Robotics and Artificial Intelligence. I am really drawn to the potential of Artificial Intelligence nowadays, and its roots point towards mathematical computations. Academically, I have maintained a good performance.

Previously, I have worked as an AI Research Intern at Planto AI, a co-pilot builder for enterprises. Here, I learned the basics of API, Integration of LLMs, and Deployment of models in virtual machines like AWS EC2.

I have learnt programming languages such as C, C++, Java, Python, Javascript, and TypeScript. Currently, I have a good hold over Javascipt, C, and Python, I have also explored web development with MERN Stack and machine learning with Python and libraries.

Timezone

Indian Standard Time ( IST ), UTC+5:30

Contact details

email:[email protected],github:ShabiShett07

Platform

Windows

Editor

I prefer VSCode as a code editor, I have become comfortable with the formatting of VSCode, also being a light code editor with a bunch of extensions, helps me to code is ease.

Programming experience

My programming journey was very basic. I started with C++ programming in my 12th grade because of studying in Computer Science. I didn't learn much about that. Then I started exploring Python as it was booming in the AI field and could handle a bunch of other use cases. There I learnt about file handling as well as some amount of machine learning and their libraries. There, I came to know about the importance of mathematics in the computing world.

In the 1st year of under studies, I learnt the basics of C programming, which didn't fascinate me much at that time. Then I started learning JavaScript and one of its frameworks MERN( MongoDB, ExpressJS, ReactJS, Node.js ), where I learnt web development. I was more into frontend than backend because I was drawn to the visual aspects of coding.

But then in the 2nd year of Under studies, my thought process changed due to more alignment of my undergraduate subjects towards mathematical computation. This made me explore the foundational structures in computation, which led me to stdlib.

I have worked as an AI Research Intern at Planto AI, which helped me to learn the basics of deployment in virtual machines, integration of LLMs etc.,

Currently, I am using Javascript as a major language along with C, which allows faster computation and reduces memory usage. Though I didn't learn FORTRAN in my journey, I have spent time understanding the structure of FORTRAN.

JavaScript experience

The first place I learnt JavaScript was through a LinkedIn course, then I started exploring JavaScript for the purpose of web development and started learning MERN stack. I didn't learn much from there, but I learnt the basics that how JavaScript alone can be a powerful programming language. Also, this was the first time I came across the working of API Calls. But still my JavaScript was not good enough that I could understand the functioning just by looking at the code.

After that, when I was practicing more on Javascript, I started contributing to stdlib. This really enhanced my hold and understanding of Javascript as most of my contributions are based JavaScript. I could start to understand the code structure and flow clearly.

The thing that I like about Javascript is its variety of frameworks. This is the reason why Javascript is widely used with different use cases.

The one thing that made me step back from javascript previously was that it has a bunch of confusing keywords and classes. This was a difficult task for me. But overall, Javascript is one of the great things that the programming world has.

Node.js experience

As I have said that I have explored some aspects of Node.js during the period of learning MERN stack. But I was not able to understand it completely.

The actual implementation of node.js I did was through stdlib. Here, I understood the clear structure of the files and the potential of the structured implementation of APIs.

C/Fortran experience

C is the language that is traditionally taught to most of the Indian undergraduates. I started it through my college itself. Actually, I didn't take much interest in considering it as a mid-level language.

But after understanding that the C can help in building a strong foundation of the codebase, making it faster, I started taking an interest in it. The only way I implemented it was by solving some basic Data Structures and Algorithm problems.

But again, I started making the best use of C once I started contributing to stdlib

Honestly, I heard about FORTRAN for the first time in stdlib. Still, I didn't make any contributions in FORTRAN but I am able to understand the code. Which is a great improvement for me. I am also planning to make a FORTAN contribution to make my understanding stronger.

Interest in stdlib

The first thing that comes to mind when I hear about stdlib is:

#include <stdlib.h>

Though it is a C standard library. This was the main line which we needed to write in our college before writing the actual C program. I never thought about why we would use this library, I used to do it as a procedure. But I understood the functioning of a library, the complexity in the backend, only because of stdlib. stdlib being my first open-source contribution platform, filled me with a dense amount of knowledge in programming as well as in mathematics.

My interest in stdlib from the day I read the repository till today has increased exponentially because the various aspects of stdlib teach me a new thing every day. This feels like a video game for me where I want to reach every corner of it. For the past few months it has been a part of my life to open stdlib check for what is new and explore more.

There are a bunch of things that I started loving in stdlib once I started contributing. The people here are keen to help each other, highly active and all serving a common goal. Though I was a complete beginner, the community never made me feel that I was new in the room.

The 2 things I like about stdlib are:

• Recently, I discovered the full functionality of repl(read-eval-print loop) and the way a whole tutorial is shown in the terminal with a clear idea is amazing
• Here, we need to go through a bunch of tests whenever we make a contribution, which makes this repo highly secure and stable. Also, we have a set of commands to test all the aspects individually, making the contributors to debug easily.

stdlib really changed the way I look at programming.

Version control

Yes

Contributions to stdlib

Merged PR's

Open PR's

Issues

I have a bunch of closed PRs because 2 of them were my initial PRs and they were a complete mess, and some of the PRs where potential duplicates, so I decided to close them instead of creating a burden in that particular issue.

stdlib showcase

Github link

Goals

The goal of the project will be to implement most of the BLAS routines into /blas/base/.

A small breakdown about this:

• Implement the C/Fortran implementation, which has prior Javascript implementations.
• Implement the remaining JavaScript implementation and their C/Fortran implementation too.
• Implement the WebAssembly packages for the completed base implementations in /blas/base/wasm.

Why this project?

Firstly, it was a high-priority project in stdlib, and I wanted to make some meaningful contributions to the community.

Secondly, being a Robotics background student, we were involved in a lot of calculations that use matrices, so these contributions will be an extension for the mathematical foundations of my career.

Thirdly, the logical thinking used in this project is quite a high-level one. This enhanced my logical thinking ability.

When I started contributing to the BLAS routines, I did not understand any logic, how different matrix calculations can be solved in a package. When I learned about matrix manipulation in college and saw this in stdlib, it created more excitement in me to wander around on the bridge of mathematics and programming.

Qualifications

I have made some meaningful contributions to the blas/base as well as blas/base/wasm. One thing I want to mention is that there were no implementations related to banded matrix, and I had figured out how to make a contribution for blas/base/dgbmv. Also, after having several discussions about the project with maintainers, I could make a blueprint of how we should approach this project.

Outside stdlib, matrix has an important place in my studies, most of the calculations in robotics are carried out by matrix, so I am directly connected to BLAS.

In this journey, to make my foundation strong as I could, I tried to refer to videos related to BLAS on the internet, and it gave me the overview of all the naming conventions, level distributions etc.,

Prior art

Talking about the history of BLAS implementations in stdlib. The tracking issue for the implementation of BLAS routines was opened by @aman-095, and he has made the major contribution to BLAS in stdlib's repository. Also, many other contributors have done tremendous work on this project to bring it this far. Currently, Level 1 of the project for double precision and single precision has almost been implemented, except for Javascript and C/FORTRAN implementations for srotmg and drotmg, also C/FORTRAN for srotg and drotg.

The references I will be using for blas/base to make contributions:

• Netlib BLAS
  This is the basis on which we are currently working.
• OpenBLAS
  This is also a useful repository. Also, I have found OpenBLAS has been used in some places of stdlib

Commitment

For the past few months, I have paused all my other commitments( like internships and projects) by completing them and giving them a proper exit, and went all in to contribute to stdlib. Still, I have no other commitments, which makes me a free bird for stdlib. I am willing to commit 30+ hrs/ week engaging in all sorts of work that will be considered as a contribution to stdlib making it my first priority for the upcoming days.

Also, I am willing to keep contributing to stdlib if it is necessary, or the stdlib community is willing for me to continue with their journey

Schedule

Implementation Blueprint

To make the implementation structure more precise, I have made a structured plan to implement the remaining /blas/base packages:

stdlib-blas-bluprint.pdf

Assuming a 12 week schedule,

• Community Bonding Period:

• Week 1: In this period, I will be engaging in getting to know my mentors, building good communication with them which will be a foundational part of a successful GSOC period. I will also be explaining my blueprint to the mentors so that they are aware of my approach, and also get their feedback and alter my plan accordingly to make it more goal oriented.

Also, during this period, I want to complete a set of work that is basic so that we maintain a clean structure before actual implementations:

1. Checking the PRs that need very small changes:
   Pull Request #4633 and Pull Request #4773 have correct implementations for zdotu and cdotu respectively, but are failing with lint error
   Pull Request #4710 and Pull Request #5022 implementation of srotmg and drotmg need to have 100% coverage of test cases

Note: There may be some more changes I need to address. I will parallelly carrying this out the whole GSoc period.

2. Addressing the PRs that need updated JSDoc as per the stdlib standards

3. Adding some WebAssembly implementations

• Week 2: Begin with Phase 1 of my implementation and get initial feedback from the mentors to maintain a good procedure in the upcoming development period.

In this phase, I want to carry out some discussions with the mentors about what steps can be taken to increase the potential of the project.

• Week 3: I will be continuing with Phase 1, as well as getting a final feedback from the feedback before the actual development period

• Development Period:

• Week 1: Finishing of the Phase 1

• Week 2: Beginning the Phase 2

• Week 3: Continuation of Phase 2

• Week 4: Completion of Phase 2

• Week 5: Beginning the Phase 3

• Week 6: (midterm) Continuing the Phase 2 and getting the feedback from the mentors, ensuring I am on the right path and taking suggestions so that the upcoming period will have a better pace and structure.

• Week 7: Completiong of Phase 3

• Week 8: Beginning of Phase 4

• Week 9: Continuation of Phase 4

• Week 10: Completion of Phase 4

• Week 11: Heading to the wrap-up of the project, considering improving the existing contributions and feedback

• Week 12: Completion of the wrap-up of the project

• Final Week: Submission of the project, along with adding required steps to implement the remaining parts of the project. Lastly, gathering feedback and guidance to enhance the development procedures and planning for future improvements.

Note:

1. The above schedule breakdown is done by taking stdlib-blas-blueprint.pdf as the reference
2. In the implementation structure, the WebAssembly implementation has not been included as it is a comparatively low-priority one, which will be carried out in parallelly during this period.
3. I have tried to give each phase a 3-week period, to be more realistic. Still, I a trying to figure out the more realistic and optimistic ones. I am open to feedback on this.
4. Phase 5 is not mentioned in the schedule because I couldn't place it realistically in the given period. My plan is to implement till Phase 4 as soon as possible and then continue with Phase 5 too.

Notes:

• The community bonding period is a 3 week period built into GSoC to help you get to know the project community and participate in project discussion. This is an opportunity for you to setup your local development environment, learn how the project's source control works, refine your project plan, read any necessary documentation, and otherwise prepare to execute on your project project proposal.
• Usually, even week 1 deliverables include some code.
• By week 6, you need enough done at this point for your mentor to evaluate your progress and pass you. Usually, you want to be a bit more than halfway done.
• By week 11, you may want to "code freeze" and focus on completing any tests and/or documentation.
• During the final week, you'll be submitting your project.

Related issues

GSoc Issue #36
Issue #2039

Checklist

• I have read and understood the Code of Conduct.
• I have read and understood the application materials found in this repository.
• I understand that plagiarism will not be tolerated, and I have authored this application in my own words.
• I have read and understood the patch requirement which is necessary for my application to be considered for acceptance.
• I have read and understood the stdlib showcase requirement which is necessary for my application to be considered for acceptance.
• The issue name begins with [RFC]: and succinctly describes your proposal.
• I understand that, in order to apply to be a GSoC contributor, I must submit my final application to https://summerofcode.withgoogle.com/ before the submission deadline.

[ShabiShett07]

CC: @kgryte, @Planeshifter, @gunjjoshi

[kgryte]

@ShabiShett07 Thanks for opening this RFC. A few comments/questions:

1. I'd prioritize real Level 3 routines over complex number Level 2 routines. The real-valued routines are more broadly useful.
2. blasjs shouldn't be used as a reference implementation. NETLIB BLAS and OpenBLAS are the two primary sources of truth.
3. While level 2 and level 3 BLAS Fortran implementations are currently blocked, pure C implementations are not and can be done. This should allow WebAssembly implementations of those level 2/3 routines with pure C implementations. This is, as you note, lower priority, however, and can be done as time allows.

[ShabiShett07]

@kgryte Thank you for your valuable feedback and I have considered all your feedback. Is there anything I need to look at?

[ShabiShett07]

gsoc-stdlib-proposal.pdf

This is the proposal I am going to submit in GSoc

[ShabiShett07]

The objective of this proposal is to continue adding BLAS (Basic Linear Algebra Subprograms) routines and their C, Fortran, JavaScript, and WebAssembly implementations. The BLAS Routines are divided into 3 levels: Level 1 (vector-vector operation), Level 2 (vector-matrix operation), Level 3 (matrix-matrix operation). I am proposing to add JavaScript, C, free-form FORTRAN implementations as well as WebAssembly Implementations. For each package, my goal will be thorough benchmarks, test cases that cover every aspect of the implementation, examples, and a README containing API signatures in multiple languages, along with the ndarray for a smooth user experience. I will be adding a docs folder that will contain a repl.txt file, a typescript file that provides introductions, usage instructions to aid in documentation, and other files to check cases where the function may become invalid. These implementations are highly prioritized because they act as dependencies for the implementation of LAPACK routines such as dgetc2, dgbtf2 etc., as well as strided statistical functions and many more. This can be helpful to explore higher-level linear algebra-based operations. A comprehensive set of BLAS routine APIs will be integrated into stdlib, with the combination of C, Fortran and WebAssembly implementations focused on optimizing performance and enabling potential hardware-accelerated execution.

This will be the description for the GSoc proposal

CC: @kgryte, @Planeshifter

[kgryte]

This project aims to increase BLAS (Basic Linear Algebra Subprograms) support in stdlib via the development of JavaScript, C, Fortran, and WebAssembly implementations capable of execution in both client- and server-side web applications. BLAS routines are divided into 3 levels: Level 1 (vector-vector operations), Level 2 (vector-matrix operations), Level 3 (matrix-matrix operations). While stdlib currently supports most Level 1 operations, several Level 2 and 3 routines involving specialized matrix storage types are currently missing. I aim to address this gap, with a particular emphasis on those routines which are key dependencies of LAPACK routines, such as dgetc2, dgbtf2, and others. A key outcome of this project will be a set of performance-optimized APIs for lower-level linear algebra operations which will subsequently help facilitate the development of high-performance machine learning and statistical analysis libraries and applications supporting execution on the web.

[kgryte]

cc @ShabiShett07

[ShabiShett07]

Thank you @kgryte, I will be using this as the project description

This project aims to increase BLAS (Basic Linear Algebra Subprograms) support in stdlib via the development of JavaScript, C, Fortran, and WebAssembly implementations capable of execution in both client- and server-side web applications. BLAS routines are divided into 3 levels: Level 1 (vector-vector operations), Level 2 (vector-matrix operations), Level 3 (matrix-matrix operations). While stdlib currently supports most Level 1 operations, several Level 2 and 3 routines involving specialized matrix storage types are currently missing. I aim to address this gap, with a particular emphasis on those routines which are key dependencies of LAPACK routines, such as dgetc2, dgbtf2, and others. A key outcome of this project will be a set of performance-optimized APIs for lower-level linear algebra operations which will subsequently help facilitate the development of high-performance machine learning and statistical analysis libraries and applications supporting execution on the web.