Writing unit tests was already a headache for developers, and AI is making it worse. A recent study has unveiled a critical weakness in LLMs: their inability to create accurate unit tests. 

While ChatGPT and Copilot demonstrated impressive capabilities in generating correct code for simple algorithms (success rates ranging from 63% to 89%), their performance dropped significantly when tasked with producing unit tests which are used to evaluate production code.

ChatGPT’s test correctness fell to a mere 38% for Java and 29% for Python, with Copilot showing only slightly better results at 50% and 39%, respectively.

According to a study published by GitLab in 2023, automated test generation is one of the top use cases for AI in software development, with 41% of respondents currently using it. However, this recent study is now questioning the quality of those tests. 

A fullstack developer named Randy on Daily.dev forum mentioned that he had tried AI for both writing code and writing unit tests, and it failed miserably as it does not understand testing frameworks like Groovy and Spock.

If you practice TDD, LLMs shouldn’t be writing your tests (as good as it seems). You think through the use cases, break down the logic, and write the tests. Let the LLM fill in the implementation.

I wonder how far someone could get with this approach

— Andrew Nguonly (@andrewnguonly) June 2, 2024

Reason Why AI is Poor at Software Testing

AI-generated tests often lack the necessary context and understanding of specific requirements and nuances of a given codebase. Due to this, AI may result in an increase of “tautological testing” – tests that prove the code does what the code does rather than proving it’s doing what it’s supposed to do.

“It’s already a danger when you write the implementation first; AI is only going to make it worse,” a user explained in the Reddit discussion.

Moreover, relying on AI for test writing can lead to a false sense of security, as generated tests may not cover all critical scenarios, potentially compromising the software quality and reliability.

When an AI is asked to write unit tests for code that contains a bug, it typically doesn’t have the ability to identify that bug. Instead, it treats the existing code as the “correct” implementation and writes tests that validate the current behavior – including the bugs, if any.

Instead, the developer says that a better use for AI would be to ask it, “What are all the ways that this code can fail?” Instead of having it write tests, have it identify things you might have missed.

Another report by researchers from the University of Houston, suggested similar numbers as ChatGPT-3.5. Only 22.3% of generated tests were fully correct, and 62.3% were somewhat correct. 

Besides, the report noted that LLMs struggle to understand and write OpenMP and MPI unit tests due to the inherent complexity and domain-specific nature of parallel programming. Also, when provided with “too much” context, LLMs tended to hallucinate, generating code with nonexistent types, methods, and other constructs.

“Like other LLM-based tools, the generated tests are a “best guess” and developers shouldn’t blindly trust them. In many cases, additional debugging and editing are required,” said Ruiguo Yang, the founder of TestScribe. 

When developers consider making new test cases, AI still has a hard time doing that. With their creative problem-solving skills, human testers still need to make thorough test plans and define the overall testing scope.

But What is the Solution?

To solve this problem, researchers from the University of Houston used the LangChain memory method. They passed along smaller pieces of the code as a guide, allowing the system to fill in the rest, similar to how autocomplete works when you’re typing.

This proves that one of the most effective ways to tackle this problem is providing more context to the AI models, such as the full code or associated libraries, which significantly improves the compilation success rate. For instance, with ChatGPT, the increase was from 23.1% to 61.3%, and for Davinci, it was almost 80%.

In recent times, tools like Cursor are helping developers build code without any hassle and in future, we might see these tools building better unit tests along with production code. 

But for now, while AI can generate tests quickly, having an experienced engineer will remain crucial to assess the quality and usability of AI-generated code or tests.

The post Why AI Can’t Get Software Testing Right appeared first on AIM.

Do we even need to learn coding anymore? The question sounds very timely as tools such as Cursor and Claude Artifacts are making everyone build apps without writing a single line of code. Cursor, which is also basically a glorified fork of the VS Code IDE, is making developers wonder if this transition to building apps in natural language marks the end of traditional IDEs.

Register for NVIDIA AI Summit India

IDE, or integrated development environment, is a code editor that allows developers to write, test, and debug code, combining multiple tools and features into a single environment. IDEs are essential for using programming languages and making useful software. 

The most famous IDE, VS Code, also allows developers to edit code such as IntelliSense code completion. However, with AI editors like Cursor, Zed, Magic, Codeium, or the most recent one Melty, integrating traditional IDEs into a developer’s workflow seems to have become redundant. 

Or Has It?

There was a recent surge of developers actively uninstalling VS Code because of Cursor IDEs. But it is easily possible for VS Code to simply add an update of AI-assisted coding onto its platform, which would eventually mark the end of Cursor. Some people also predict that Microsoft might simply end up acquiring Cursor in the future. 

John Lindquist, creator of egghead.io, said that he had a chat with Harald Kirschner, project manager at VS Code, about Cursor and VS Code recently, and the team is keenly aware of Cursor’s capabilities and there might be several things in the pipeline. “I think we’ll all be pleasantly surprised,” he said.

Other IDEs such as Jetbrains, PyCharm, and IDLE, are also facing a similar crisis as AI-generated coding gains traction. 

Regardless, modern generative AI coding tools are capable of integrating several open-source LLMs instead of a traditional IDE like VS Code, making it handy for several AI developers. “You can select code and ask questions based on that piece of code. So you don’t have to keep switching between the IDE and browser,” explained a developer on X.

But it simply means that most of the IDEs in the future would come with generative AI integration. It would become the default for IDEs as it was for integrating no-code and low-code platforms. 

Moreover, though it is becoming easier for non-developers to build apps, the task of building high-end software is still far away from these autocoding platforms. These tools can enable non-developers or those with minimal coding experience to create apps without needing to interact with an IDE, but those cannot be replicated everywhere. 

When it comes to experienced developers, AI tools can fasten the prototyping process by generating samples of code through prompts to quickly test ideas. But in the long run, when it comes to dealing with complex, customised, and large-scale projects, traditional IDEs still are better at debugging and handling crucial features.

The Future of IDEs is AI-Assisted

Before talking about the end of IDEs, it is essential to understand what they stand for. With IDEs, developers have a standardised environment with a defined codebase for meeting specific requirements. This consistency of code cannot be fully controlled by an automated code generation platform like Cursor, or others. 

On the contrary, future IDEs could take this further by generating larger chunks of code, or even entire modules, based on brief descriptions or prompts. The future of software development is likely to see AI-integrated IDEs becoming the norm. This would be helpful in analysing code in real-time and automatically correcting them. 

Furthermore, with generative AI integrated into IDEs, these code editors would be able to suggest context-aware code, which is not just based on the syntax, and even optimise it. This would also enable a personalised developer style of coding, enabling natural language prompting with coding, much like Cursor and Claude. 

AI-integrated IDEs will likely combine traditional development tools with generative coding, making software development more efficient, intuitive, and accessible while still providing the depth needed for complex projects. This would enable developers who know coding to take generative tools to the next level. 

But at the same time, it would be difficult to manage generative AI code if it is available within the IDEs as people still would be hesitant to trust generative AI for boilerplate code. It could also result in the loss of coding skills for skilled developers. 

The post Will AI Coding Tools Mark the End of IDEs? appeared first on AIM.

The struggle is real. When a developer uses multiple package managers, there’s a risk of modules being overwritten or conflicting. Nearly 4.71% of packages on PyPI have module conflicts in their dependency graphs, leading to broken environments or even security risks.

Often, different package managers use different lock file formats, which can cause issues when switching between tools or collaborating with others using different package managers. In Python, things can get much worse when you consider dependency management.  

In the Python world there are multiple package managers including pip, Conda, Poetry, pipenv, pyenv which seem to have their own flaws. 

Why it matters? This makes it confusing for both new as well experienced developers, and eventually things feel unreasonably slow. Most users try to solve this by replicating other’s environment without giving it a second thought, and that does not work either.

But, Python is deaf for dependency resolution 

One of the primary issues in Python dependency management is handling conflicting dependencies. 

For instance, pip, the default package manager, cannot handle two different versions of the same package. This situation, often touted as “dependency hell”, causes entire installations to fail, leading to unexpected behaviour in projects.

A few months ago, one of the Reddit users mentioned that Python really feels like a programming environment from the early 80s, where a developer had a single project on their pc, and that was all they worked on for years. 

“Python wants to do everything on the global system level, including runtime versioning and packages. That means that any two developers can think they have a working project on their system, even though they have radically different setups. This makes handing off and deploying Python applications a nightmare,” he added further, suggesting why dependency resolution is a nightmare on Python. 

However, the most important and weird part of the dependency resolution is that pip makes assumptions. The pip documentation on dependency resolution explains that pip makes assumptions about package versions during installation and later checks these assumptions, which can lead to conflicts if the assumptions are incorrect.

Managing dependencies can be resource-heavy. One user reported having about 100+ GB of their hard drive filled with Python virtual dependencies, highlighting the storage impact of multiple environments.

Ergo, Virtual Environments

“I’m afraid of having 2000 folders, each one with a different virtual environment,” said one Reddit user expressing confusion about virtual environments. Running a project solely or in isolation becomes cumbersome. 

While virtual environments are essential for project isolation and dependency management, there are instances where users find virtual environments problematic rather than solving the problem. 

Previously, users have reported that package versions and dependencies can still conflict within virtual environments, requiring manual resolution in some cases that directly question the isolation in Python. 

Some developers view virtual environments as wasteful, believing they unnecessarily duplicate libraries for each project. As one Reddit user stated, “It seems like you’re installing a new copy of every library every time you start a new project, which seems like a waste of resources.”

The complexity of virtual environments can be overwhelming for those new to Python. A Reddit user expressed extreme frustration, saying, “I spend way more time just trying my computer to get my virtual environment up, project dependencies installed, and IDE configured than I do actually coding.”

Several developers recommend using Docker to avoid virtual environment issues altogether. This approach encapsulates the entire environment, making it more reproducible across different systems.

The post Python Can’t Get Package Management Right appeared first on AIM.

Python has long been celebrated for its simplicity and readability. However, for developers coming from languages like C++ or Java, this syntax can sometimes feel unfamiliar or challenging. This problem served as an inspiration for Bython, funnily touted as the Python with braces. 

Bython is a Python preprocessor which aims to provide an alternative syntax for Python that uses curly braces to define code blocks, similar to languages like C++ or Java, while still leveraging Python’s interpreter and ecosystem.

But it does not end there. The main aspect of Bython is you don’t have to worry about indentation. It won’t give you errors even if you mess up tabs/spaces or copy one piece of code to another that uses a different indentation style; it won’t break.

Javascript for Python?

Python usually gets a lot of hate for whitespaces as when you accidentally mix tabs and spaces, it gives you an error which is hard to find. A Reddit user mentioned that he loves brackets as it does not break while copying. “I spend more time tabbing than I do putting two brackets and auto formatting,” he added further. 

The key thing to note here is that Bython uses Python for interpretation, meaning existing Python modules like NumPy and Matplotlib still work seamlessly.

Jagrit Gumber, a frontend developer on X, while praising Bython, mentioned that Bython is really worth it if you come from a C C++, C# or Java background. 

Some developers prefer using braces in code because it allows them to freely cut, paste, and rearrange code sections, relying on automatic formatting tools to correctly align and indent the code afterwards, regardless of how it was initially typed.

It is also related with muscle memory of programmers who are coming from other programing languages. When you spend enough time with one programing language, you develop a habit of using curly braces and most programming languages do use braces. A developer on Hackernews mentioned that he had developed a muscle memory of using braces by pressing Ctrl + M and he was able to parse code much faster when it is both indented and with proper brackets.

Bython also allows developers to write code that is both easy to read and write, like Python, while also being able to leverage the efficiency and speed of C. “Bython provides easy ways to call C functions and leverage C libraries, more control over hardware, and performance optimisations,” said Saikat Sinha Ray while explaining how Bython can be used to call C functions. 

Some users loved Bython so much that they wanted all its features to be integrated into Python itself. A user on Hackernews mentioned said, “This shouldn’t be “Bython”, it should be Python 4.0,” suggesting the next iteration of Python 4.0 should have Bython baked into it. 

Will Bython Replace Python?

While Bython solves some key issues which bothers multiples users, it can not replace Python by any means. Bython is more like a passion project from Mathias Lohne and it is not meant to be integrated into the Python project as not everyone finds the whitespaces an issue. 

There are users who want to use Python and have no issues with its syntax. There are even users who hate the idea of using braces and find the syntax of Python better than ever. So we would suggest to think of Bython as an extension of Python which is optional and can be used if you have issues with braces and whitespaces.

The post Meet Bython, the Python With Braces appeared first on AIM.

A few months ago, NVIDIA open sourced their drivers, starting with R515 for Linux. This solves one big problem of getting up and running a Linux system powered by an NVIDIA card. However, with users requesting this for years, what made NVIDIA finally open source them?

The answer is simple. Most developers use Linux and drivers are the most stressful part of using the NVIDIA GPU. 

Not just developers, you will even find many big tech companies, including OpenAI and Google, rely on Linux to build their AI platform. 

TensorFlow, one of the most popular AI development libraries by Google, is best compatible with Ubuntu, a well-known Linux distribution. On Windows, however, you’ll need to use WSL (Windows Subsystem for Linux), which essentially acts as a virtual machine for running Linux.

Even Google DeepMind, one of the leading AI research labs, previously used a modified version of Ubuntu, called Goobuntu, but later switched to Debian testing. These are both Linux-based operating systems. 

Further, IBM’s Watson, known for its natural language processing and machine learning capabilities, runs on SUSE Linux Enterprise Server. 

CUDA Performs Better on Linux

It might come as a surprise, but many users have reported that CUDA performs better on Linux than Windows. 

Mike Mikowski, the manager of the Kubuntu Focus project, mentioned that NVIDIA GPU drivers are almost not faster on Windows, especially for deep learning solutions which use CUDA and OpenCL interfaces. 

NVIDIA has been using Ubuntu exclusively to demonstrate deep learning on all their edge solutions, which suggests Linux performs better for deep learning tasks compared to Windows. 

Meanwhile, a Reddit user reported that when he used CUDA on the same hardware but with different operating systems, including Windows and Ubuntu, the latter performed better. 

A reason behind this performance gain is Linux has a better GPU command scheduling than Windows.

Linux Makes AI Dev Environment Setup a Breeze

Despite the fact that Linux has a limited choice of software, you will find that a majority of AI developers use Linux. The key reason behind this is how well the software libraries are available and work well with Linux. 

For example, getting CUDA and CuDNN up and running is a hassle and not a seamless process on Windows. Compared to that, on Linux, everything can be managed via package manager without any issues. 

This is one of the reasons why most software development libraries and tools make their way to Linux first. 

A Reddit user, while answering why everyone uses Linux, mentioned that installing and setting up development software on Linux is a breeze thanks to a healthy array of package managers. 

“On Ubuntu or Debian-based systems, apt has most of what one will need to set a machine up. That, coupled with Conda, pip, and perhaps some additional software… it’s fast to install new libraries and programs, and dependencies are handled automatically,” he added further. 

Linux being a versatile operating system matches the target production system in most cases as most production workloads are based on Linux. A user on Stack Overflow mentioned that CUDA performs better on Linux, stating, “If you are looking at a performance point of view, and the time taken for a build, it would be best to use Linux.”

When you combine better performance with CUDA and the ease of configuring the development environment, that too with other benefits like security, large community and support, there’s no doubt that Linux is powering the AI moment.

The post How is Linux Powering the AI Moment? appeared first on AIM.

Web application developers, who want to use LLMs, naturally turn to TypeScript.

TypeScript, as a programming language, is a great fit when it comes to creating fast, user-friendly AI applications. Its asynchronous programming features let multiple tasks run concurrently, which is key when dealing with potentially slow AI model calls. This means that the application can stay responsive even if AI is working in the background. 

As a result, there’s been a lot of discussion around whether TypeScript will replace JavaScript for AI development.

Why TypeScript Matters

One of the key reasons why developers are turning to TypeScript for AI is its ability to catch errors at compile-time rather than run-time. 

With complex AI algorithms and large datasets, identifying and fixing errors early in the development process is crucial. TypeScript’s static typing helps ensure code quality and reduces the likelihood of bugs slipping through to production.

A recent JetBrains survey reveals that TypeScript is rapidly gaining adoption, with 28% of developers now using it regularly. This growing popularity can be attributed to the language’s ability to improve the overall developer experience and increase confidence in the code.

Harrison Chase, the co-founder and CEO of LangChain, highlighted the growing trend of AI development happening in TypeScript. “We are seeing a lot of AI development happen in TypeScript. Evaluation is a CRITICAL part of AI development! Being able to evaluate gives you the confidence to move fast,” he added. 

While sharing a post-mortem report on previous bugs and issues at JSConf Hawaii, 2019, Brie Bunge, a senior staff software engineer at Airbnb, confirmed that 38% of bugs at Airbnb could have been prevented using TypeScript.

Apparently, TypeScript’s compatibility with popular AI libraries like TensorFlow.js and Brain.js enables developers to leverage existing JavaScript tools and frameworks while benefiting from TypeScript’s type safety and enhanced developer experience. 

This compatibility allows for a seamless integration with the vast JavaScript ecosystem.

Moreover, the introduction of TypeScript-first AI development frameworks like TypeAI and Axilla.io are good examples of the community’s commitment to making TypeScript a first-class citizen in the AI ecosystem. 

These tools provide developers with the necessary abstractions and utilities to build AI applications more efficiently and with fewer errors.

Apart from AI development, TypeScript, being a simple language, can also help you understand concepts. “TypeScript made me understand what I’m actually doing with every function, variable, before writing it… Also, it’s easier to understand object structure and just know more about certain objects,” a Reddit user said.

Can it Replace Python? 

Short answer: No. Python can not be replaced with any programming language for AI and ML development – at least for now. 

“Python has a lot of use cases around data vis, ML/DS, CI/CD, general scripting, and other things, some of which TS/JS just isn’t really used for currently,” said a Reddit user, suggesting a large ecosystem for data science and machine learning tasks.

We can’t deny Python’s dominance in AI, but there are users who have used both and prefer TypeScript. 

A Reddit user in favour of TypeScript said, “Most folks here are pretty adamant that Python is the only way. I spent years coding in Python (mostly Django API servers and some Pytorch) and have now spent a couple of years coding in TS, including building RAG backends. I dramatically prefer TS for a few reasons: Package management with npm is much better…The TS type system is much better than Python types and more widely supported by packages.”

Ultimately, while Python currently remains the dominant language for AI development, TypeScript is gaining traction and offers a compelling alternative for certain use cases. So it will be interesting how the TypeScript community in future will make efforts to make it more relevant for AI development.

The post Why are Devs Turning to TypeScript for AI Development? appeared first on AIM.

Microsoft is not shy of giving developers what they want. After Excel, and possibly Windows as well, the company introduced Copilot with GitHub. But somewhere there, VS Code, which has one of the largest footprint as a software for developers in today’s world, got lost for the recognition of its contribution.

Register for NVIDIA AI Summit India

“Not sure if Microsoft can truly comprehend the true footprint of VSCode,” said Jaana Dogan, principal engineer at Google. But at the same time, there is an apparent spree of people not liking what VS Code has become, stale. Though the abundance of repositories on GitHub that are forks of VS Code is undeniable, the new tools in the market are making VS Code struggle.

“Just uninstalled VS Code,” said a developer on X. Why? Probably because of the release of Cursor AI, which is touted as the ChatGPT moment in coding, is like the final nail in the coffin of VS Code. But is it really the end of VS Code?

What are the Problems?

There are several problems with VS Code, which is undeniable. Mohamed Yamani, a front end software engineer, said that VS Code for Python development sucks, explaining that it was not highlighting the problems with his code. Developers in the thread agree that they often end up using other IDE’s such as Jetbrains or even vim for Python. 

Even for C# and other languages, though VS Code has been introducing several updates, developers are not happy. Also, it is very easy to install malicious extensions through VS Code.

Coming to Cursor, which is basically a glorified fork or extension of VS Code, is capable of integrating several open source LLMs instead of an IDE like VS Code, making it handy for several AI developers. “You can select code and ask questions based on that piece of code. So you don’t have to keep switching between the IDE and browser,” explained a developer on X.

VS Code works well, but when it comes to AI development, it falls majorly behind Cursor with the integration of Claude 3.5 Sonnet, and other LLMs. Though VS Code allows integration of other LLMs such as Phi-3.5 or GPT-4, Cursor is just way more flexible with the availability of LLMs including Llama 3.1.

Alex Sidorenko, a React developer, made a video comparing VS Code with Copilot vs Cursor using Claude 3.5 Sonnet for Next.js App Router, where Cursor felt more aware and easier.

Cursor’s coding and AI capabilities should be a wake up call for Microsoft to make VS Code integration with GitHub Copilot a lot easier. “My assumption is that Cursor’s sudden surge in popularity will wake Microsoft up and they’ll make VS Code/GitHub Copilot integration a lot better, landing in a few months,” said Jamon Holmgren, the founder of Infinite Red.

Is it too late for Microsoft’s VS Code or GitHub Copilot to catch up? 

Cursor is Just One VS Code Update Away from Extinction

Exactly a year ago, Cursor was touted as if VS Code and ChatGPT had a baby. Cut to today, it is being called as a replacement for VS Code and GitHub Copilot. Conversations around dropping VS Code started exactly a year ago with the introduction of Cursor. But VS Code is still going strong. 

In a Reddit discussion, as developers were discussing the capabilities of Cursor and VS Code, they were willing to bet big on VS Code in the long run, as Cursor built by a small startup would struggle to compete with Microsoft. “Even if Cursor survives through the next year, I would go with VS Code just to have something standard and mainstream to compare it to,” said a user.

John Lindquist, creator of egghead.io, said that he had a chat with Harald Kirschner, project manager at VS Code, about Cursor and VS Code recently, and the team is keenly aware of Cursor’s capabilities and there might be several things in the pipeline. “I think we’ll all be pleasantly surprised,” he said. 

It seems like a Cursor like update for VS Code is coming soon, which might possibly make Cursor obsolete. Or to survive, Cursor might end by getting acquired by someone like OpenAI or Anthropic. 

But when it comes to VS Code, given the massive footprint it has, plans in the pipeline, it is hard to just uninstall it and move on. The concept of making everyone a developer using natural language, is not unknown to Microsoft. 

The post Why Developers are Uninstalling VS Code appeared first on AIM.

Unlike typical software development, LLM development is a distinctly different and more complex task, with its own unique set of challenges. One of the most formidable challenges faced by LLM developers is the “curse of multilinguality”. 

Sara Hooker, VP of research at Cohere AI, said, “When you try and make AI actually work for the world, you’re talking about this vast array of different languages. There are 7,000 languages in the world, and 80% of those have no text data.” 

This lack of diverse language data leads to models that overfit high-resource languages like English and Chinese while under-serving the “longtail” of low-resource languages.

It doesn’t stop at that. It gets worse for the reasoning part.

The Elusive Nature of Reasoning

As Subbarao Kambhampati, professor at Arizona State University, illustrates with the classic “manhole cover” interview question, “The tricky part about reasoning is if you ask me a question that requires reasoning and I gave an answer to you, on the face of it, you can never tell whether I memorised the answer and gave it to you or I actually reasoned from first principles.”

Assessing whether an LLM can truly reason rather than just match patterns is difficult. There is often a gap between an LLM’s ability to generate code or text that looks plausible versus its deeper understanding of the underlying logic and ability to reason about it.

Natural language relies heavily on context, shared understanding, and inference to convey meaning. This makes it difficult for LLMs to extract precise semantics and formal logic needed for rigorous reasoning just from language examples.

Furthermore, LLMs have no concept of reality outside of language and cannot test the truth of statements. They are unconcerned about whether concepts contradict each other and only focus on generating sentences that follow language rules.

David Ferrucci, the founder of Elemental Cognition, argues that natural language is insufficient for reliable logical reasoning and computations in complex domains. He states that “for complex reasoning problems where you cannot afford to be wrong, natural language is not the right medium”. 

“Without any underlying formalism, natural language’s ambiguity and subjectivity are great for casually navigating around into another human’s brain, but not the best for ensuring shared meaning and precise, reliable outcomes,” he added.

Ferrucci suggests that formal languages and reasoning systems are needed to enable complex problem-solving.

The Verification Gap

Perhaps the most critical challenge that LLM developers face is the lack of robust methods for verifying the outputs of these models. As Kambhampati notes, “It’s very hard to show what is and what is not on the web,” making it difficult to determine whether an LLM’s output is grounded in factual knowledge or mere hallucination.

A research paper titled ‘TrustLLM: Trustworthiness in Large Language Models’ developed a trustworthiness evaluation framework examining 16 mainstream LLMs across eight dimensions, including fairness, machine ethics, privacy, robustness, safety, truthfulness, accountability, and transparency. 

The researchers found that none of the tested models was truly trustworthy according to their benchmarks, highlighting the need for improved verification methods.

Aidan Gomez, the CEO of Cohere, mentioned that to improve reasoning, language models need to be shown how to break down tasks at a low level, think through problems step-by-step, and have an “inner monologue”. 

“However, data demonstrating this type of reasoning process is extremely scarce on the internet,” he added. 

One of the most significant challenges in verifying the outputs of LLMs is their inherent “black box” nature. LLMs are complex, opaque systems that make it difficult for developers and researchers to understand how they arrive at their outputs.

LLMs suffer from a lack of interpretability, which means it is challenging to understand the reasoning behind their responses. This opacity makes it difficult to identify the root causes of incorrect or inconsistent outputs, hindering efforts to improve the models’ reliability.

Another related issue is the limited explainability of LLMs. Even when an LLM provides an answer, it is often unclear how it arrived at that particular response. This lack of explainability makes it challenging for developers to troubleshoot issues and refine the models.

Addressing the challenges faced by LLM developers will require a multifaceted approach. This includes developing more advanced verification methods to assess the factual accuracy and logical consistency of LLM outputs, improving the interpretability and explainability of LLMs to better understand their inner workings.

By focusing on these key areas, researchers and developers can work towards creating LLMs that are more reliable, trustworthy, and capable of complex reasoning across diverse languages and domains.

The post What is Stopping Devs from Building an LLM? appeared first on AIM.

While graph neural networks (GNNs) have long been the preferred choice for graph-based learning tasks, they often rely on intricate, custom-designed message-passing mechanisms. 

However, a novel approach has emerged that offers comparable or superior performance with a more streamlined methodology.

Enter masked attention for graphs (MAG), an innovative technique that reimagines graph representation. MAG conceptualises graphs as collections of nodes or edges, maintaining their interconnectedness through clever masking of the attention weight matrix. 

This simple solution has proven remarkably effective, surpassing not only robust GNN benchmarks but also more sophisticated attention-based approaches across a diverse array of over 55 node and graph-level challenges.

Overcoming Traditional Problems

Junaid Ahmed, a software engineer, mentioned in a recent post on LinkedIn that there are certain limitations of transitional GGNs, including complexity in design, scalability issues, and performance variability, and they can be solved through MAG. 

Apparently, MAG does not use any positional encodings, which is in contrast to current trends for GNNs and Transformers. It also does not require sophisticated learning rate schedulers or optimisers.

A professor of Beijing University of Posts and Telecommunications who goes by fly51fly on X mentioned MAG scales sub-linearly in memory and time with graph size. “It also enables effective transfer learning through pre-training and fine-tuning,” he said. 

Another research paper titled ‘Masked Graph Transformer for Large-Scale Recommendation’ proposes using a Masked Graph Transformer (MGFormer) architecture for efficiently capturing all-pair interactions among nodes in large-scale recommendation systems.

The research paper further mentioned that the MGFormer outperforms GNN-based models, especially for lower-degree nodes, indicating its proficiency in capturing long-range dependencies for sparse recommendations. 

Even with a single attention layer, the MGFormer achieves superior performance compared to baselines.

Expanding the Horizons of Graphs via Masking 

In the field of cybersecurity, the cybersecurity entity alignment via masked graph attention networks (CEAM) model leverages an asymmetric masked aggregation mechanism to address the unique challenges of aligning security entities across different data sources.  

By selectively propagating specific attributes between entities, CEAM significantly outperforms state-of-the-art entity alignment methods on cybersecurity-domain datasets.

Masking has also found its way into video event recognition through the masked feature modelling (MFM) approach. 

MFM utilises a pre-trained visual tokeniser to reconstruct masked features of objects within a video, enabling the unsupervised pre-training of a graph attention network (GAT) block. 

When incorporated into a state-of-the-art bottom-up supervised video-event recognition architecture, the pre-trained GAT block improves the model’s starting point and overall accuracy.

In the domain of temporal knowledge graph reasoning, the Attention Masking-based Contrastive Event Network (AMCEN) employs historical and non-historical attention mask vectors to control the attention bias towards historical and non-historical entities. 

By separating the exploration of these entity types, AMCEN alleviates the imbalance between new and recurring events in datasets, leading to more accurate predictions of future events.

Overall, the emergence of MAG and related masked attention-based techniques represents an exciting new direction in graph representation learning, offering both simplified architectures and state-of-the-art performance across a wide range of applications like cybersecurity, reducing biases and video event recognition. 

The post Even Your Graph Neural Networks Need Attention appeared first on AIM.

It has been crazy how AI tools are making everyone a developer. Earlier it was GitHub Copilot, though its allure among developers has slowly been dying. Now, it is Cursor AI, which is now being celebrated as the best developer tool for AI right now.

To give an example, Ricky Robinett, VP of developer relations at Cloudflare, posted a video of his eight-year-old daughter building a chatbot on the Cloudflare Developer Platform in just 45 minutes using Cursor AI, documenting the whole process, even the spelling mistakes while giving prompts!

With the latest updates, Cursor AI included the feature of an alternate composer window that gives suggestions based on queries for the next step. People have been building apps in just a couple of days without even writing a single line of code. 

According to developers, the code completion quality and speed of Cursor AI is oftentimes better than GitHub Copilot and Supermaven AI, as it comes with Copilot++. For flexibility, the AI editor also allows users to switch from Microsoft’s VSCode into Cursor anytime, as it’s built on top of it. 

It has been only a few months since Cursor emerged out of nowhere, and people are still trying to figure out how it works so fast. Taking to X, Jack Zerby, a designer for Gumroad, said he was able to build an equity calculator with the help of Cursor AI, which would have taken weeks using other tools.

Compared to GitHub Copilot, which charges $10 per month after a one month free trial, Cursor AI is free for most use cases. But for the Pro version, it costs $20 per month with unlimited completions and 500 fast premium requests using GPT-4 or Claude-3.5 Sonnet. This is comparatively expensive when compared to GitHub Copilot, which has unlimited requests to OpenAI’s Codex Model.

Taking the Crown of the Fastest Copilot?

Cursor AI is being built by a team of developers constituting Anysphere, that started only two years back. It raised a seed funding of $8 million from the OpenAI Startup Fund, which also included participation from former GitHub CEO Nat Friedman and Dropbox co-founder Arash Ferdowsi. Cut to the present, Cursor AI is emerging as the strongest competitor to GitHub Copilot. 

According to reports, the team, led by Michael Truell, Sualeh Asif, Arvid Lunnemark, and Aman Sanger, is also in talks to raise over $60 million in its series A round, co-led by a16z and Thrive Capital, with Patrick Collison, the co-founder of Stripe also participating. This will get Anysphere $400 million post money valuation.

The Cursor AI tool has been adopted by several engineers at companies such as Samsung, Replicate, Midjourney, Shopify, and Perplexity. It can also access the existing codebase of teams, which helps it in retrieving accurate code in clicks, or just by hitting tab, along with natural language prompts and instructions.

People have been experimenting in different ways with the AI tool, and found that it works the best when it is paired with Anthropic’s Claude 3.5. Jordan Singer from Figma said that Cursor is definitely showing what the future of development will be. “All this talk of Claude + Cursor and becoming capable of building anything you put your mind to (no matter your skill set) is warranted. If this is the future, I want to live in it,” he said. 

When Santiago Valdarrama, founder of Tideily, posed the question, “If you aren’t using GitHub Copilot to write code, can you tell me why?”, one of the most common reasons given was that developers have started using Cursor AI, which Valdarrama said is a very genuine reason. 

Is Everyone Finally a Developer?

When Satya Nadella announced that everyone’s a developer with the rise of coding tools that can help code in natural language, he was mostly talking about GitHub Copilot and ChatGPT. But Cursor AI has taken it a step further because of its flexibility, even giving tough competition to Cognition Labs’ Devin. 

But it’s not just Cursor AI which is changing the landscape of coding. In April, Eric Schmidt-backed Augment also came out of stealth with $252 million, gaining a valuation of $977 million. Built by Ex-Microsoft software developer Igor Ostrovsky, Augment is similar to Cursor, utilising open tools available in the market. 

And apart from this, there is a dearth of coding Copilots in the market from Magic, Tabnine, Codegen, TabbyML, and even Amazon CodeWhisperer. But right now, people are in love with Cursor AI, even after using it just for 10 minutes.

The post Cursor AI is Better at Coding Than GitHub Copilot Will Ever Be appeared first on AIM.

Everyone’s a developer because of tools like ChatGPT and Claude. Similarly, when GitHub Copilot was introduced, it was hailed as a game-changer, capable of speeding up coding tasks, reducing repetitive work, and even generating code snippets based on natural language prompts. However, the reality for many developers has been less than ideal.

To understand the reasons, Santiago Valdarrama, an ML teacher and founder of Tideily, asked the question – If you aren’t using GitHub Copilot to write code, can you tell me why? And the replies were mixed and nothing short of fascinating.

Tom Cruickshank, a developer who tested Copilot with JetBrains IDE, shared his experience. “I tried it out with Jetbrains IDE and I felt it was more of a miss than a hit. I was deleting stuff more than using what it was giving. And I could not justify the price for the value I was getting. Tried it for a month. Removed it, Cancelled subscription.” 

Cruickshank’s frustration highlights a common sentiment among developers who feel that the tool’s output often misses the mark, leading to more work rather than less. Developers often spend more time debugging the code than actually using it within production. 

Another primary complaint from experienced developers is that Copilot disrupts their workflow. Darren Shepherd, an accomplished coder, pointed out, “For languages I am very experienced in, it interrupts my mental flow with long suggestions. I don’t really need help on what to do, I know what I’m doing, I need help on names/references that I don’t remember. 

For seasoned professionals, these interruptions can be more of a hindrance than a help, breaking the focus needed for complex problem-solving.

While Copilot might be helpful for simpler tasks, many developers find it lacking in more complex scenarios. Mark, another developer, commented, “LLMs are great for a particular type of coding, not all, by far.”

“I Don’t Know Coding”

The use of Copilot also raises concerns about the learning process for new developers. Ross Murphy expressed a cautionary note: “If you ever want to create anything new, you’re going to need to understand how to actually write code, and not just use a glorified copy-paste tool. 

Using GitHub Copilot is one sure-fire way to never actually learn how to do coding. 

Developers emphasise the importance of maintaining a clear mental model of their code. Copilot is not very useful for anything beyond auto complete. As a programmer, you want to have full context in your head of how a chunk of code works. Sometimes, Copilot suggests something more advanced and ends up in the waste of several hours. 

Similarly, AI tools like Copilot are known to sometimes generate incorrect or nonsensical code, a phenomenon known as “hallucination.” Han MF Brolo simply stated, “It hallucinates too much.” This issue can be particularly frustrating, as it forces developers to spend additional time reviewing and correcting code that should have been accurate from the start.

And apart from the hallucinations and wrong answers, many developers are sceptical about actually using the code within their production due to copyright issues. Copilot, trained on a vast amount of public code, might sometimes suggest code with security flaws or outdated practices, posing risks if used without proper review.

Additionally, since Copilot is trained on public code, it might generate snippets that resemble copyrighted or licensed code, potentially causing legal issues if used inappropriately, particularly in commercial projects.

It may also generate code that doesn’t match a team’s coding standards or architecture, leading to inconsistencies and requiring extra time for refactoring and review. People have been frustrated with this and have slowly stopped adopting such AI tools. 

Not For Everyone

Given these challenges, many developers are turning to alternatives or choosing to code without AI assistance altogether. Some have found more satisfaction with other tools: “I use Cursor. I’ve tested both, and I must admit that Cursor is much better. Since I started using it, I’ve been more productive, I manage to do a lot more in a workday. It’s impressive.” 

This suggests that while Copilot may not be the perfect fit for everyone, other tools might better align with certain workflows and preferences. Some people simply want chatbots like ChatGPT or Claude to generate the whole code as they don’t know what they are doing in the first place. 

For some developers, the decision to use or avoid Copilot boils down to its value proposition. As Mike Rybka explained, “Used it for about 6 months. In the end, I felt like it threw me out of the flow too often. Even though 90% of the time the suggestions were accurate, you still have to double-check everything anyway. I find it easier to just type out exactly what I want, even if it takes longer.” 

The need to verify Copilot’s output can negate any time savings, making it less appealing for those who prefer to work with precision.

Ultimately, whether or not to use GitHub Copilot depends on individual preferences, the nature of the coding tasks, and the level of expertise.

The post Why Developers Are Ditching GitHub Copilot appeared first on AIM.

In the world of shell scripting, Bash has long been the go-to language for automating tasks and glueing together various command-line tools. However, Bash’s quirks and limitations have left many developers yearning for a more modern and expressive alternative. 

Enter Amber, a new programming language that compiles to Bash, bringing the power of high-level programming to the realm of shell scripting. 

“Amber is not a wrapper of Bash as implementing new data types and error handling won’t be possible using a wrapper“, said Paweł Karaś, creator of Amber, in an exclusive interaction with AIM.

High-Level Scripting Language

Though Amber operates similar to Bash, it is designed for frequent command execution.

“Amber provides built-in error handling capabilities, making it easier to write robust scripts,” said Karaś. 

The new programming language is designed specifically for the scripting style where executing external commands and manipulating their results is integral. 

In Python, executing a command and capturing its output requires several lines of code and explicit error handling. Amber, on the other hand, provides a concise and intuitive syntax for command execution, along with built-in error handling capabilities. 

One of the standout features of Amber is its focus on runtime safety. Almost all operations available in the Amber language are safe. 

The exceptions are commands that refer to the Bash shell, which can fail, and failing functions that can fail at any time through error propagation or by explicitly using the fail keyword.” 

Amber requires developers to handle errors whenever a command or failing function is called. This can be done by propagating the error upwards, handling it in a dedicated failed block, or marking the command as unsafe if the developer is certain it will succeed. 

“This approach ensures that scripts written in Amber are more robust and less prone to unexpected failures,” Karaś emphasised.

Long Way to Go 

One concern among potential Amber users is the performance impact compared to vanilla Bash. As there’s no way to handle float values in Bash, Amber has to rely on bc a calculator for Bash, which means that Amber requires running an external program every time to perform an arithmetic operation. 

The team behind Amber is planning to create a new type of data called Int that will use Bash’s built-in ability to do math with whole numbers (integers). As Bash can’t handle decimals, they will also look into better ways to deal with Num, which is the data type Amber uses for numbers with decimals.

Meanwhile, Amber’s syntax has raised some eyebrows, particularly the use of > instead of # for comments. Karaś clarified the reasoning behind this choice, stating, “Amber is a language designed to resemble the syntax of ECMAScript, where comments start with //”

The idea behind this choice is to give users the feeling of writing in a language that looks familiar. This decision aligns with Amber’s goal of providing a modern and intuitive programming experience for Bash scripting.

A Reddit user asked why Amber is not directly compatible with POSIX shell, which is a standardised command-line interface and scripting language that ensures compatibility and portability across different Unix and Unix-like operating systems, especially in the era of the minimal container images where POSIX shell can have the lightest posix-compliant shell possible.

“We are considering creating a compilation flag for maximum compatibility, such as --output=sh,” Karaś explained. “Amber would then return errors when syntax unavailable for the given format is used. This idea is being developed and may evolve into a better solution.” 

Karaś also touched on Amber’s ability to handle spaces in file names, a common pain point in Bash scripting. He assured us that Amber’s outputted code will soon be Shellcheck verified, ensuring proper handling of file paths and other string interpolations.

As promising as Amber’s potential seems, one must remember that it is still in the alpha stage, and it is not recommended for use on production servers. Improvements are still being made and developers are currently working on a standard library so as to not worry about interpolating file path to open a file. 

The post ‘Amber’ is Here to Take on ‘Bash Scripting’ appeared first on AIM.

Databricks acquiring Tabular was the talk of the Bay Area at the recent Data + AI Summit.

Whether by coincidence or not, the announcement was made during Snowflake’s Data Cloud Summit, held last week.

Databricks chief Ali Ghodsi definitely has some answers, or maybe not? 

“Now, at Databricks, we have employees from both of these projects, Delta and Iceberg. We really want to double down on ensuring that Delta Lake UniForm has full 100% compatibility and interoperability for both of those,” Ghodsi said, admitting they don’t understand all the intricacies of the Iceberg format, but the original creators of Apache Iceberg do. 

Talking about Databricks’ mission to democratise data and AI, Ghodsi started his keynote by saying that every company today wants GenAI but at the same time everybody is worried about the security and privacy of their data estate which is highly fragmented. 

He pointed out that the data estate of every company is placed into several data warehouses and the data is siloed everywhere. This ends up bringing a lot of complexity and huge costs to the companies and ultimately gets them locked into these proprietary system silos. 

Databricks’ Delta Lake Project (+ Apache Iceberg) to the Rescue!

With a focus on addressing these issues, Databricks announced the open-source Delta Lake Project a few years back. 

Ghodsi explained that the idea was to let users own their data and store it in data lakes where any vendor can then plug their data platforms into that data, allowing users to decide which platform suits them best. This removes lock-in, reduces the cost, and also lets users get many more use cases by giving them the choice to use different engines for different purposes if they want. 

“This was our vision and we almost succeeded but unfortunately there are now two camps. At Databricks we have Delta Lake, but a lot of other vendors are using this other format called Apache Iceberg,” said Ghodsi.

Delta Lake and Apache Iceberg emerged as the two leading open-source standards for data lakehouse formats. Despite sharing similar goals and designs, they became incompatible due to their independent development.

Over time, various open-source and proprietary engines adopted these formats. However, they typically adopted only one of the standards, and frequently, only aspects of it. This selective adoption effectively fragmented and siloed enterprise data, undermining the value of the lakehouse architecture.

Now, with the Tabular acquisition, Databricks intends to work closely with the Iceberg and Delta Lake communities to bring interoperability to the formats themselves as highlighted by Ghodsi. 

Tabular, Inc, a data management company was founded by Ryan Blue, Daniel Weeks, and Jason Reid. Blue and Weeks had developed the Iceberg project at Netflix and donated it to the Apache Software Foundation. 

As the largest contributor to Iceberg, Tabular is seen as the company driving Iceberg, playing a key role in advancing Iceberg within data management frameworks. 

“I’ve known Ryan for a long time. We worked closely with him when he was back at Netflix, and some of the team members were working with him even before that when he was at Cloudera. So it’s been a very close collaboration,” Ghodsi said. 

Databricks’ UniForm, now generally available, offers interoperability among Delta Lake, Iceberg, and Hudi. It supports the Iceberg restful catalogue interface so that companies can use their existing analytics engines and tools across all their data. 

Furthermore, with the inclusion of the original Iceberg team, the company plans to expand the scope and ambitions of Delta Lake UniForm. It envisions bringing both the Delta Lake and Iceberg formats closer in the future to a point where their differences won’t matter, according to Ghodsi.

So basically, with the Tabular acquisition, Databricks seems to be trying to build this pendrive or USB port of sorts that can be plugged into AI systems in the future — achieving 100% interoperability.

“It will simplify the developer experience and allow them to move up the stack in the value chain. As, instead of worrying about which version of Iceberg or Delta they are using, developers can be rest assured that all of that is solved through the UniForm format,” said Databricks’ vice president of field engineering APJ Nick Eayrs in an exclusive interview with AIM. 

Eayrs explained that with this, developers will now be able to spend more time on analysis, enrichment, and transformation of the data rather than worrying about version conflicts. 

“Our commitment is reinforced to open source. We have open-sourced our Unity Catalog, and we continue to build the default open standard when it comes to the data format,” he added. 

The Other Side of the Acquisition 

The tabular acquisition development came just after Databricks’ competitor Snowflake announced it had adopted Apache Iceberg tables as a native format and also introduced Polaris, a catalogue for Iceberg tables accessible by any data processing engine that could read the format, such as Spark, Dremio, and even Snowflake itself.

In addition, Microsoft announced an expanded partnership with Snowflake. As part of this, Microsoft Fabric’s OneLake will now support Snowflake’s Iceberg tables and facilitate bi-directional data access between Snowflake and Fabric.

Databricks’ decision to acquire Tabular was spurred by customer demand for better interoperability among data lake formats. People have also weighed in on the significance of Databricks’ Tabular purchase in light of Snowflake’s activity.

While the acquisition of Tabular indicates that both Databricks and Snowflake are positioning for the influence of AI on data infrastructure, the purchase has clearly put new pressure on Databricks’ competitors including Snowflake. 

When asked if Databricks is planning to work closely with Snowflake to bring Iceberg and Delta Lake together, Ghodsi added, “Governance in open-source projects involves working closely with the community and those that have committers in the project. And if some of these committers happen to be employed by Snowflake, we’ll work with them as well.”

The post Databricks is Taking the Ultimate Risk of Building ‘USB for AI’  appeared first on AIM.

There has been a lot of buzz around the newest programming language, Bend. Discussion forums have been pitting it against CUDA, the go-to choice for experienced developers. However, with CUDA’s restrictions and worthy alternatives, Bend could be worth the excitement. 

Bend is a high-level, massively parallel programming language designed to simplify parallel computing. Unlike traditional low-level languages like CUDA and Metal, Bend offers a Python-like syntax that makes parallel programming more accessible and easy to developers without deep expertise in concurrent programming.

“Bend automatically parallelises code, ensuring that any code that can run in parallel will do so without requiring explicit parallel annotations. As such, while Bend empowers developers with powerful parallel constructs, it maintains a clean and expressive syntax,” Vinay Konanur, VP – emerging technologies, UNext Learning, told AIM.

Why Not Cuda, Then? 

One might wonder how it measures up against low-level languages like CUDA. While CUDA is a mature, low-level language that provides precise control over hardware, Bend aims to abstract away the complexities of parallel programming.

Bend is powered by HVM2 (Higher-Order Virtual Machine 2), a successor of HVM, letting you run high-level programming on massively parallel hardware, like GPUs, with near-ideal speedup.

A user mentioned that Bend is nowhere close to the performance of manually optimised CUDA. “It isn’t about peak performance,” he added.

Bend is based on the Rust foundation, which means you can expect top-notch performance through simple Python-like syntax. Konanur also revealed that Bend’s interoperability with Rust libraries and tools provides access to a rich ecosystem. 

“Developers can leverage the existing Rust code and gradually transition to Bend,” said Konanur. 

Moreover, he believes that the performance of a programming language on a specific GPU can depend on several factors, including the specific GPU, the nature of the task, and how well the task can be parallelized. 

“So, even if Bend were to support AMD GPUs in the future, the performance could vary depending on these factors,” Konanur added. 

Scalability and Parallelisation 

Bend’s official documentation suggests that as long as the code isn’t “helplessly sequential”, Bend will use thousands of threads to run it in parallel. User demos have proved the same. 

A recent demo showed a 57x speedup going from 1 CPU thread to 16,000 GPU threads on an NVIDIA RTX 4090. This is a perfect example of how Bend runs on massively parallel hardware like GPUs and provides near-linear speedup based on the number of cores available. 

Focusing on parallelisation, Bend is not limited to any specific domain, like array operations. It can scale any concurrent algorithm that can be expressed using recursive data types and folds, from shaders to actor models.

Max Bernstein, a software developer, argues that Bend has different scaling laws compared to the traditional languages. While Bend may be slower than other languages in single-threaded performance, it can scale linearly with the number of cores for parallelisable workloads.

How about Other Programming Languages? 

A Reddit user, when asked how different Bend is from CuPy or Numba, answered, “It massively reduces the amount of work you need to do in order to make your general purpose program parallelisable, whereas CuPy and Numba (as far as I know) only parallelise programmes that deal with multidimensional arrays.”

Further, users have also observed that Bend is not focused on giving you peak performance like the manually optimised CUDA code but rather on simplifying code execution by using Python/Haskell-like code on GPUs, which wasn’t possible earlier. 

When you compare Bend with Mojo, a programming language that can be executed on GPUs and provides Python-like syntax, Bend focuses more on parallelism across all computations. Mojo is geared more towards traditional AI/ML workloads involving linear algebra.

But unlike Mojo, Bend is completely open-source which means users can take and modify the code as per their convenience. Also, they can contribute to the project as it ensures more transparency. 

The post Bend It Like Python, Scale It Like CUDA appeared first on AIM.

CUDA is a wonderful piece of tech that allows you to squeeze every bit out of your Nvidia GPU. However, it only works with NVIDIA, and it’s not easy to port your existing CUDA code to other platforms.

You look for an alternative to CUDA, obviously. 

What are the alternatives to CUDA?

1. OpenCL: An open standard for parallel programming across CPUs, GPUs, and other processors with some performance overhead compared to CUDA.
2. AMD ROCm: An open-source GPU computing platform developed by AMD that allows the porting of CUDA code to AMD GPUs.
3. SYCL: A higher-level programming model based on C++ for heterogeneous processors enabling code portability across CUDA and OpenCL through Intel’s DPC++ and hipSYCL.
4. Vulkan Compute: It is a compute API of the Vulkan graphics framework, enabling GPU computing on a wide range of GPUs with lower-level control.
5. Intel oneAPI: It is a cross-architecture programming model from Intel, including a DPC++ compiler for SYCL, offering an alternative to CUDA for Intel GPUs.
6. OpenMP: It is an API for parallel programming on CPUs and GPUs. It uses compiler directives, and recent versions support GPU offloading as an alternative to CUDA.

Let’s address each with more depth.

1. OpenCL

OpenCL (Open Computing Language) is an open industry standard maintained by the Khronos Group that lets you utilise parallel programming across various platform architectures. 

OpenCL allows you to write a program once, which it can then run on several different processors from different companies like AMD, Intel, and NVIDIA.

This can be useful if you want to use the hardware you already have or if you want to choose the best processor for a specific task, regardless of which company made it.

2. AMD ROCm

ROCm (Radeon Open Compute) is a platform designed by AMD to run code effectively on AMD GPUs. But the best part is that ROCm is open-source and can be accessed by everyone.

One of the most important parts of ROCm is called Heterogeneous-computing Interface for Portability, or HIP. HIP is quite close to CUDA programming in terms of syntax. This means if you know how to program CUDA then there’s no stiff learning curve if you’re switching over. 

There’s even a tool called HIPIFY that can automatically convert CUDA code into code that works with HIP and AMD GPUs, with just a few minor changes required.

3. SYCL

SYCL (pronounced “sickle”) is a higher-level programming model based on standard C++ for heterogeneous processors. SYCL is built on top of the C++ programming language, enabling code portability across OpenCL devices.

The core idea of SYCL is to provide the performance of OpenCL with the flexibility of C++. Good examples of SYCL include Intel’s DPC++ (Data Parallel C++) based on Clang/LLVM that can target CUDA and OpenCL devices.

4. Vulkan Compute

Vulkan’s low-overhead design and close-to-metal nature can enable performance close to and sometimes even exceeding CUDA in many compute workloads. It provides compute shaders to enable GPU computing.

Since Vulkan Compute is a relatively new technology, its ecosystem is still maturing in terms of libraries, tools and language binding. It has a steeper learning curve, especially when graphics interoperability is also used.

However, new Vulkan Compute-focused frameworks like Kompute are emerging to make Vulkan GPU computing more accessible.

While Vulkan Compute can also interoperate with APIs like OpenCL, CUDA and DirectX 12, there are some very specific features like CUDA’s dynamic parallelism, that are not available with Vulkan.

5. Intel oneAPI

oneAPI is an open, unified programming model developed by Intel that aims to simplify development across diverse computing architectures (CPUs, GPUs, FPGAs, and other accelerators).

oneAPI consists of a core set of tools and libraries, including DPC++ language and libraries for deep learning, machine learning and more.

A key goal of oneAPI is to provide an alternative to proprietary models like NVIDIA’s CUDA. It aims to prevent vendor lock-in and allow code portability across Intel, NVIDIA, AMD and other hardware.

Furthermore, case studies have shown up to an 18x speedup for compute-intensive algorithms using oneAPI tools and Intel hardware.

6. OpenMP

Open Multi-Processing, or OpenMP, is an API that supports multi-platform shared-memory parallel programming in C, C++, and Fortran. It has also been used for parallel computing in CPUs.

Recent versions of OpenMP, starting from version 4.0, have introduced support for GPU offloading. This allows OpenMP to be used for GPU computing as an alternative to CUDA.

OpenMP provides a higher level of abstraction compared to CUDA. It handles many low-level details like data movement and kernel launches automatically, which can make it easier to use for some developers. 

CUDA is a proprietary solution from Nvidia and it is fine-tuned to get the most out of Nvidia hardware. So, finding an exact replacement may not be possible as they’ll always have an advantage over any open-source platform. Sure, if you want to run parallel computation over other GPUs, then the given solution will get the job done in the most efficient way possible. 

The post Top 6 Parallel Computing Alternatives to CUDA appeared first on AIM.

The internet is buzzing, and once again, Python finds itself in the crosshairs. A user sparked a lively debate with her post on X: “Hating on Python is literally a skill issue.” This simple statement set off a firestorm of opinions.

The point, as succinctly captured in the phrase, suggests that those who criticise Python might be struggling with the language’s simplicity and accessibility. It’s reminiscent of the common gripe about C++: “C++ is terrible because of memory leaks.” 

Python, with its straightforward syntax and wide applicability, is easy to pick up but deceptively challenging to master in nuanced, efficient coding.

Some argue that “Python is easy to hate on because it’s a trash language that’s merely an abstraction anyway.” This critique is not uncommon. 

Many developers accustomed to the power and precision of languages like C++ or Java often find Python’s abstract nature frustrating, especially when dealing with performance-intensive applications.

But What is the Issue?

Sanjay Nithin offers a more technical criticism saying, “Can’t handle environments properly, properly handled environments aren’t easy to deploy, takes a lot of time and space to process.” These are legitimate concerns, especially in production environments where efficiency and scalability are paramount.

Some people have positive criticism about Python and say that lack of constraints such as typing encourages bad practices throughout the ecosystem. 

This highlights a frequent complaint among seasoned developers that Python’s flexibility can lead to sloppy code and poor software engineering practices if not managed properly, which has been a long discussed issue.

Conversely, Andre Infante argues that programming languages are designed to minimise errors: “The whole point of programming languages is to protect you from your own mistakes and make it harder to make them. Skill issues are language issues.” 

The ease of making mistakes in Python is a double-edged sword—it makes the language accessible but also more prone to user error.

“It is hard to beat Java at speed. I’m an ML engineer, and Python is slow. Do write Java whenever you can.” This critique underscores Python’s performance limitations compared to more optimised languages like Java.

AGI will be built on Python

Despite these criticisms, supporters of Python are unwavering: “Seems 100% true, Python is based and will be based.” This sentiment reflects the strong community support and the language’s entrenchment in various tech domains, particularly AI and data science.

Proficiency and comfort with Python come from skill and familiarity, not inherent flaws in the language. Earlier, AIM made a point that the most frustrating programming language is also the one that you work on the most. 

As the market shows, Python’s popularity is no fluke. It’s a testament to its broad applicability and the skill of those who wield it effectively. According to Statista, Python is the most popular programming language.

Python’s dynamic typing can lead to subtle bugs that may not be caught until runtime, and its performance can sometimes lag behind that of compiled languages like C++. Apart from people heavily discussing how dynamic typing is one of the worst things about Python, it is also the community that makes it worse for people.

“Python is great for scripting random tasks and calling functions implemented in C. It is not great for building applications.” This pragmatic take acknowledges Python’s strengths and weaknesses, recognizing its utility in specific contexts.

In the end, hating Python may indeed be a skill issue. “Only a bad craftsperson blames their tools.” Python’s simplicity and versatility have made it a staple in the programming world. While it may not be perfect for every application, its accessibility and vast library support make it an indispensable tool for many.

Meanwhile, this debate seems to be forgetting the fact that English is the hottest programming language. Moreover, Microsoft is now also allowing people to code in their native languages, which could even mark the end of Python. The conversation of ‘skill issue’ is simply going to shift to whoever prompts the Copilot the best.

The post Hating on Python is Literally a Skill Issue appeared first on AIM.

Recently, AIM reviewed the best tools to run large language models (LLMs) locally on a computer, and Ollama stood out as the most efficient solution, offering unmatched flexibility. Ollama, an open-source tool developed by Jeffrey Morgan, is revolutionising how enthusiasts run LLMs on their local terminals. 

With its user-friendly interface and compatibility with popular models like LLaMA 2 and Mistral, Ollama can easily be considered by those who wish to experiment with LLMs securely, cost-effectively, and efficiently. It enables users to harness the power of advanced AI models without relying on cloud services or expensive hardware. 

AIM tested Ollama by running multiple LLMs across multiple operating systems, including Linux (Pop!_OS), macOS, and Windows, to give readers a comprehensive overview of this utility.

Ollama: A Blazing-fast Tool to Run LLMs Locally

While it stood out as the fastest solution to run LLMs locally on a terminal, if you are not comfortable with that, there’s also a provision of GUI, but it requires some additional steps (that most basic users would want to avoid).

Ollama has access to a wide range of LLMs directly available from their library, which can be downloaded using a single command. Once downloaded, you can start using it through a single command execution. This can be quite helpful for users whose workload revolves around a terminal window. If they are stuck somewhere, they can get an answer without switching to another browser window. 

While using Ollama on Linux, we found a rather surprising approach taken by its developers. When you execute the installation script, it takes care of all the GPU drivers by itself, with no extra steps required. 

On the other hand, we found macOS’s installer to be the most refined among the three platforms, as it was pretty easy to navigate and use. Sure, you eventually have to switch to the terminal, but the way it takes you to the terminal was smooth. 

However, if you wish to use Ollama with a GUI, you can use Open Web UI. But it can only be installed through a Docker container, which can be troublesome for users not familiar with the concept of containerisation. 

AIM noted that Ollama’s only downside was that it did not provide official documentation on how to use the already downloaded LLMs. 

Model Matters the Most

Most certainly, using a minimal utility like Ollama to run LLMs locally can give you a little edge over other tools, but in the end, what model you are using matters the most.

For example, we used Llama 2 and Llama 3 side by side on Ollama, and we were surprised with the results.

As you can see, Llama 2 started answering our question within a few seconds, whereas LLaMA 3 took long but gave compelling and detailed answers to the given criteria. We tested LLaMA 2 on multiple utilities, but Ollama + terminal gave the fastest results, which is why we concluded that Ollama is the fastest when used in the terminal.

Also, if you try loading a very large model beyond what your specs can handle, like any other tool, Ollama will not be able to load the given model. And the sad part is it does not even inform you that Ollama has stopped loading the model. 

The only way to tackle this problem is to keep an eye on system resources. When you see a sudden drop in resource consumption, it indicates Ollama has failed to load the model, and you can stop the entire process (or else it will show the loading animation endlessly).

All in all, we found Ollama flexible and fast. It lets you use GUI if you want to and gives you blazing-fast responses when used in a terminal, making it a win-win situation. However, if you want to use something simple, we’ll soon introduce you to Jan, a tool that lets you run LLMs locally with the least effort possible. 

The post Why Ollama is Good for Running LLMs on Computer appeared first on AIM.

Every computer nerd wants to control how his data is collected on the internet. Apart from data privacy, there are other reasons for this, such as working in isolated software environments and being less likely to connect to the internet to run LLMs. 

Sure, there are lots of open-source LLMs available these days, but they can not be directly accessed. To access, use, and modify them, one would need a platform to work with the data, and that’s where our five options come into play. 

AIM tested the listed options with multiple LLMs, checking their performance on three platforms: Linux (Pop!_OS), MacBook M1, and Windows 10, including how they behaved with different LLMs, such as their response time, formatting, and UI.

Ways to Run LLMs Locally on Your Computer

• Jan: Cleanest UI with useful features like system monitoring and LLM library.
• Ollama: Fastest when used on the terminal, and any model can be downloaded with a single command.
• llamafile: The easiest way to run LLM locally on Linux. No additional GUI is required as it is shipped with direct support of llama.cpp.
• GPT4ALL: The fastest GUI platform to run LLMs (6.5 tokens/second).
• LM Studio: Elegant UI with the ability to run every Hugging Face repository (gguf files). 

Jan: Plug and Play for Every Platform 

While testing, Jan stood out as the only platform with direct support for Ubuntu, as it offers a `.deb` file that can be installed with two clicks, with no terminal interaction required. Furthermore, its user interface is the cleanest among the listed options. 

Also, by utilising the LLM, you can access the system monitor, which shows the total CPU and memory consumption. It uses a markdown markup language to output the answer, which means each point is well structured and highlighted, which can be quite helpful when working with programming questions. 

You can also choose from a large library of LLMs, including closed-source (which requires API keys to download and use). It also lets you import LLMs, allowing you to easily load manually trained models. 

To download/learn more about Jan, visit their GitHub page.

Ollama: The Fastest Way to Run LLMs Locally on Linux 

Ollama can be used from the terminal, but if you want to access the GUI, you can use the open web UI (it requires docker). We also tried Ollama on the terminal and GUI, and the response in the terminal was close to how we interact with LLMs over the internet. 

This makes it the best option for someone who wants to run LLMs locally on Linux, as most Linux users will be interacting with the terminal. Besides, having access to lightning fast local LLMs directly in the terminal will be great news.

Another good thing about this utility is how it manages dependencies for AMD GPUs automatically on Linux. So, you are not required to download any specific dependencies on your own, it will be taken care of automatically.

Furthermore, you can directly download LLMs with a single command. Also, you can customise the prompt with multiple parameters, such as changing temperature (to balance predictability and creativity), changing system messages, etc. Later on, you can load the custom model for fine-tuned answers. 

It worked great when paired with an open web UI, but we encountered a problem when interacting with the terminal. 

In the terminal, when you try to load a model, and, for some reason, if it does not load, it won’t show you any error message and show the loading LLM message forever.

It can be tackled by monitoring system resources while loading LLM. If you see a sudden drop in resource utilisation, it means Ollama has stopped loading LLM; you may close the terminal or stop Ollama:

You can visit Ollama’s GitHub page for installation instructions and configuration options.

llamafile: Only Option to Run Huge LLMs Locally on Mid-range Devices 

When we started experimenting, llamafile was the only utility that let us use Mistral 70B on our laptop. Sure, it took a long time to generate, but the other options mentioned in the list simply could not load the model itself.

Unlike the other options, you have to download each model from its GitHub repository. This repository can be executed as a script, so you are not required to load the entire model in your RAM before execution.

Also, you are not required to install the GUI platform manually, as it comes backed up with llama.cpp to provide you with a GUI. Besides, it is hosted locally on your browser to minimise resource consumption. 

To cater to advanced users, there are many configuration options such as prompt template, temperature, Top-P and K sampling, and advanced options like how many probabilities to show and various Mirostat configuration options.

Visit llamafile’s GitHub page to download the LLM files directly.

GPT4ALL: Fastest Way to Run LLMs Locally on a Computer

GPT4ALL was the fastest utility in our testing, giving 6.5 tokens/second. We own a mid-range computer, and it is fast for the specs we own. Sure, if you were to pair it with a high-end machine, you’d get better numbers.

Also, they provide an installer for Ubuntu, Mac and Windows for seamless installation, which is a positive thing but for Linux, it does not create a desktop icon, and you end up moving to a specific directory to launch the GPT4ALL.

It has a huge library of LLMs that you can download and run locally. But there was no way to load a locally downloaded model, so you had to rely on their offerings only. Also, it was the only utility on the list that prompts users to share their data and chat responses to improve GPT4ALL. 

The only issue that we found is we were not able to use API keys to access the OpenAI GPT-4 model. Also, the user experience is cluttered as developers have added too many options. 

Furthermore, the structure of the output was given in a markdown format but printed in plain text, so users end up having markdown symbols in their plain-text outputs. 

Visit GPT4ALL’s homepage for further information.

LM Studio: Direct Access to Hugging Face Repository 

LM Studio was the only utility that did not work on Linux, and struggled to keep up when used on Mac. This means we were only able to use it smoothly on a Windows laptop. However, it can be different if you use different Linux distributions or a higher-spec version of Mac. 

The best part is that you can load and prompt multiple LLMs simultaneously with the consult access. Also, you can access any Hugging Face repository within the LM Studio itself, and if not, you can still get the desired LLM from their large library. 

Furthermore, you can use locally downloaded/modified LLMs and host an LLM on a local HTTP server. 

For further information, visit LM Studio’s official webpage.

Next up, we will explore the possibilities of running LLMs locally on Android/iOS devices to help protect your privacy on the device you use the most – your smartphones. 

The post 5 Ways to Run LLMs Locally on a Computer  appeared first on AIM.

For over a decade, a popular belief has been that a computer science degree is all you need to tread the path to wealth, especially in a country like India.  

If you looked around then, everyone and their neighbour pursued engineering college degrees to land software engineer jobs. Cut to 2024, things are slowly changing.

Check: NIRF Ranking of Engineering Colleges

“Winter is coming for software engineering,” said Deedy Das from Menlo Ventures, sharing a graph (see below) about how by 2024, software engineering roles will almost become a distant memory. 

The graph clearly shows that there was a boom in hiring tech talent back in 2021, which has now dropped to 40% of it in 2024. Added to this are the layoffs in big tech, such as the recent one where Google decided to lay off its Python programmers for outsourcing to cheaper options. 

Even startups, one of the biggest providers of jobs to developers, are now preferring to hire tenured people with expertise. “Startups often dislike hiring new grads because the cost to train them and get them up to speed is quite high,” explained Das in the post, adding that freshers with knowledge about new technologies such as AI are preferred. 

It seems like AI is stealing jobs indeed. There are a lot of CS majors who are no longer able to find jobs as now everyone wants to be a programmer. With AI tools in the market, it has become easier to become a programmer, and the barrier to entry has reduced significantly, making everyone want to do something with software. 

What about more jobs?

Das said that people would migrate from software for a little bit and then come back during the next boom cycle, just like AI. Isaac Hasson suggested developers to stop studying pure computer science and get skilled in other disciplines such as biology and chemistry, which he said are all going to transform pretty soon as well. 

This is what Yann LeCun, the chief of AI at Meta, posted on X about a year back. It takes at least 15-20 years to have an effect on productivity. The delay is determined by how fast people learn and adapt to it. “So no, AI is not going to cause instant mass unemployment,” LeCun concluded. “It is only going to displace jobs over time and make people more productive,” just like any other technological revolution.

On the other hand, several predictions suggest that there will be many more jobs in the future. According to Francois Chollet, the creator of Keras, “There will be more software engineers (the kind that write code, e.g., Python, C or JavaScript code) in five years than there are today.” He adds that the estimated number of professional software engineers today is 26 million, which would jump to 30-35 million in five years.

For context, he said that many are currently claiming that people shouldn’t get into computer science because, in the future, most of the software engineering will be done by AI, the likes of Devin, Devika, and the recent GitHub Workspace update. In the latest podcast with Lex Fridman, when asked how much programming people would do in the next 5-10 years, OpenAI CEO Sam Altman said, “A lot, but I think it’ll be in a very different shape.” 

Similarly, a user Bjoern said, “ I see AI as the fracking tech of software engineering that allows us to extract enormous amounts of previously inaccessible oil,” which he said are the long tail of software use cases that were not available before. “We will need more software engineers to decide what to build, how to scale things, and how to maintain that long tail of complexity.”

Computer science is not all you need

The number of CS majors in universities have increased exponentially over the last decade and the trend seems to increase even more. But the problem is that people are applying for software engineering roles while relying completely on AI and Copilot tools, which are not enough to get jobs in the market.

An entire generation is studying for jobs that won’t exist. Mark Cuban last year predicted that the highest-paying college major in the world, computer science, will hold very little value for employers in the future. “Twenty years from now, if you are a coder, you might be out of a job,” he said in a podcast.

One of the most important things that Altman and LeCun agree upon is that humans should be trained with AI and use them as copilots. Well, the most-celebrated phrase across 2023 has been, “AI won’t take your job, it’s somebody using AI that will take your job.” And, “you will be 10x more valuable in the coming years”, as Logan Kilpatrick said on X.

“The best practitioners of the craft will use multiple tools and they’ll do some work in natural language,” he added. Altman explained that people would be able to focus on the higher level of abstractions, and the puzzle-solving skill set of programming, which Fridman agreed, was the harder part. 

Furthermore, implying that a lot of software engineering jobs would be redundant is dangerous as many students would not know if they should attend college at all.

The post Software Engineering Jobs are Dying appeared first on AIM.

It is hard to imagine what today’s operating systems will mean to developers 37 years in the future. Going back 37 years to 1986, when MS DOS version 4.0 was released, computers were less complex and more open to modifications at a fundamental level. 

Microsoft, embracing the open source movement, released version 1.1 and 2.0 in 2014. Last week, the company, along with IBM, which was closely associated with building the 4.0 version of DOS, made the source code available on GitHub. 

The open source release also included the beta build and scanned documentation of MT-DOS 4.0 built primarily by Microsoft. MT-DOS introduced multi-tasking, which allowed certain programs to run in the program. This version didn’t see commercial release but was quickly overshadowed by OS/2 which improved upon the feature. 

Microsoft claims to have ‘lost’ its source code.

It was unearthed by Connor ‘Starfrost’ Hyde, who discovered unreleased beta binaries of DOS 4.0 among former Microsoft CTO Ray Ozzie’s collection. 

He worked to publish the code on GitHub along with Scott Hanselman, the VP of developer community, and Jeff Wilcox, head of OS programs office, who reported they will “continue to explore the archives and may update this release if more is discovered”.

In 2001, Bill Gates announced, “Today it really is actually the end of the MS DOS era.” DOS, which ran underneath Windows 3.1, Windows 95, and Windows 98, was being replaced by Windows XP. 

At the time of the announcement, DOS was running on over 400 million PCs worldwide. 

But the popularity of vintage games built on DOS has retained old timers and children. There are plenty of games still being released on DOS barring the limitations of memory management. 

Developers, who reminisced about the time they worked with DOS, were enthusiastic about experimenting with the release and found a few inconsistencies in it. 

Hackers and tinkerers who grew up with DOS are eagerly digging into the source code, reliving their early computing experiences. As one HackerNews user exclaimed, “Holy cow, this is huge! DOS 1.x and 2.x are too old to run a lot of vintage DOS software, but 4.x would run just about everything.”

Nostalgia and the present!

The average person doesn’t often come in contact with systems run on DOS. Apart from George RR Martin, who wrote A Game of Thrones books on DOS, the US Navy that used it till 2011, or old-time nerds who are the market for DOSBox to play retro video games on, the operating system is practically defunct. 

Right after the open source release, developers took to social media to discuss the times they spent working on DOS when alternatives were sparse. While modern operating systems are complex and allow a wide range of functions, they’re more closed and secure. 

A user wrote, “They (kids) can play with code in various sandboxes locally and on the web, but the computer fundamentally belongs to someone else today.” 

Although slow with less memory, the device would allow for programming in languages like C to directly compile on the device itself, and modify the core software. 

This is in contrast to many modern systems, which often restrict such low-level access to enhance security and stability.

Using the source code, one developer built his own version of the DOS. Though he mentioned the difficulties in compiling the code, he successfully demonstrated running various commands and utilities, showing the functionality of the built operating system within the FreeDOS environment.

Bryan Lunduke, a former software engineer of Microsoft, who attempted to compile the source code, found that several files, including GETMSG.ASM and USA.INF, were “mangled” and could not be properly compiled.

A software engineer and historian, Michal Necasek, who runs the OS/2 Museum blog, found that shoving the source code into GitHub caused irreversible damage, such as loss of timestamps and corruption of files that used code page 437 characters. 

Necasek argued that for practical purposes, old source files should be treated as binary and must be preserved without modification. These changes raise concerns about the integrity and historical accuracy of the released source code. 

Despite the issues with the release, many in the community are excited to dig into this piece of computing history. 

As one developer commented on HackerNews, “Even just seeing some of these filenames is a major blast from the past! Love it,” listing off familiar file names like ‘CPY.BAT’, ‘MAKEFILE’, ‘RUNME.BAT’, and ‘SETENV.BAT.’

The Future is Retro 

While the released version of MS-DOS 4.0 may not have much practical use today, the publicly available source code has allowed hackers to explore the inner workings of the operating system that defined their early computing experiences. 

Another user shared, “I never thought I would be able to build DOS from scratch, but here we are!” The real value is in preserving this important part of the history of personal computing. 

Jim Hall, the founder and project coordinator of FreeDOS, explained that the move to use the MIT licence allows developers to freely study, modify and distribute the code. 

Modern developers’ fascination with retro computing extends beyond MS-DOS, as seen in the rise of retrocomputing. One user on X posted, “We completely missed out on the days of programming in C for MS-DOS back when things were so much simpler.” 

Engaging with classic operating systems’ source code and retro media allows developers to gain insights into computing’s evolution, informing future innovations. 

The series ‘Fallout’, for instance, draws inspiration from vintage technology, MS-DOS 4.0 release exemplifies this retro revolution, inspiring a new generation to embrace the hacker legacy.

Bojan Tunguz aptly said, “The future is here and it’s totally retro.”

The post Open-Source MS-DOS 4.0 Inspires Aspiring Developers to Embrace Retro Revolution appeared first on AIM.

Suddenly, Japan seems to have become the sweet spot for tech investments. OpenAI recently opened its new office in the country and released a custom GPT-4 model for Japanese. Microsoft announced it would invest $2.9 billion over the next two years to increase its hyperscale cloud computing and AI infrastructure in Japan. 

A few days ago, Google invested $1 billion to boost digital connectivity between the US and Japan by constructing two new subsea cables, Proa and Taihei. These cables will also link to the Pacific Islands, enhancing reliability and reducing latency.

Oracle, too, has announced an investment of $8 billion over the next 10 years to meet the demand for cloud computing and AI infrastructure in Japan. Earlier this year,  Amazon Web Services (AWS) announced its plans to invest $15.24 billion in Japan by 2027 to expand cloud computing infrastructure that serves as a backbone for AI services.

Last year, CEO Jensen Huang said his company would do its best to supply its GPUs to Japan amid extremely high market demand.

And while the tech giants make a beeline for Japan, there emerges a clear winner – the Indian tech professionals. Given the demand, Japan is now actively hiring Indian professionals, particularly in the technology and engineering sectors. This trend is driven by the country’s ageing population and labour shortages in various industries.

Can India Fulfill Japan’s Needs? 

“I’m glad to see more AI in Japan,” said Llion Jones, the co-founder of Tokyo-based Sakana AI and one of the authors of Google’s 2017 research paper ‘Attention Is All You Need’. 

Further, he told AIM that they’ve interviewed a couple of Indian folks. “If you go to sakana.ai/careers, we received only about 33% of applicants from here [Japan], and the rest are from overseas. About 13% of them are from India,” he added.

According to a recent report, Japan aims to expand direct investment and attract skilled tech workers from Southeast Asia and India. Moreover, the country recently announced its plans to introduce new visas to make it easier for Indian and Southeast Asian tech workers to move to Japan. 

Japanese Ambassador to India, Hiroshi F Suzuki, extended a warm invitation to Indian students and young people, encouraging them to consider visiting and studying in Japan. 

In a conversation with Hindi-speaking YouTuber Mayo San, Ambassador Suzuki said that it is very easy to get a student visa to Japan. He said all you have to do is present a student ID card to get a visa. “I’m encouraging young Indians to go to Japan to get skill training and job opportunities,” he said. 

According to the Japanese Ministry of Justice’s 2023 statistics, 46,262 Indian nationals live in Japan, mainly in Tokyo. They primarily work in the information technology (IT) and creative sectors.

Working in Japan— a Humbling Experience 

AIM spoke with a few Indian employees working in Japan, who say that their experience has been positive. 

“By 2050, about 60% of the Japanese population will be above 50. If this happens, who’s going to generate revenue for them? Moreover, one AI can do the job of two or more people. The country is trying to address these problems,” shared an Indian employee who currently works at Rakuten. 

Further, the employee said learning basic Japanese is recommended for Indian employees. At Rakuten, however, the communication happens in English as its official language. 

The Rakuten employee also said that Japanese people are very humble and helpful. If they don’t understand English, they make the effort to translate the documents into Japanese.

Further, the employee added that if you are fluent in L3 Japanese, you can become a manager in Japan as well, but he noted that most Indians in Japan work in the tech sector. 

Interestingly, 10,000 of Rakuten’s 50,000 employees are Indians and 2,000 work in Japan. 

Rakuten recently partnered with OpenAI to create solutions to address the unique needs and challenges of telecommunications operators when planning, building and managing mobile networks. 

OpenAI backed-Speak is also pretty popular in Japan. 

Another employee who works for an IT firm said that in Japan, there is no hire and fire culture, and the Japanese Ministry looks after the workers. “Every day that you work in an organisation, your timesheet is sent to the Japanese Ministry at the end of the day,” he said.

The Land of Rising AI Indeed 

There is no stopping for Japan. The country is focusing on sovereign AI. Recently, Fujitsu Limited and Oracle partnered to offer sovereign cloud and AI capabilities, meeting the digital sovereignty needs of Japanese businesses and the public sector.

Moreover, unlike the EU and US, copyright laws in Japan regarding generative AI are quite lenient, which leads to Japan being called a ‘machine learning paradise’.

According to reports from a committee meeting, Japan’s minister of education, culture, sports, science, and technology, Keiko Nagoaka, indicated that AI companies in Japan can use “whatever they want” for AI training without restrictions based on profit motives, the nature of the activity, or the source of the content, including materials from illegal sites.

With such laws, Japan is definitely an attractive place for AI startups to train their models. Vinod Khosla said, “Japan and India are set to be the next AI hotspots”.

The post Japan is the Next Big Hub for Indian Tech Talent appeared first on AIM.

TypeScript has become ubiquitous with web development. In the past 12 years since its release, developers around the world have adopted it, as the faster and simpler version of JavaScript. 

Created by Microsoft, the language is used internally in the development of Visual Studio Code and Office 365 suite, among other products. Angular, which is Google’s open-source web application framework based on TypeScript, is widely used by YouTube, Google Cloud etc. 

Amazon also builds its business logic of AWS Construct Library packages in TypeScript, while also providing mappings into each of the supported programming languages.

To prove this in numbers, GitHub last week released the innovation graph tracking different parameters over the past four years and TypeScript went from the 11th position to the 4th as the most used language on the platform. 

Such rapid rise in popularity is also attributed to JavaScript, the language which it is the superset of. TypeScript adds static typing to JavaScript, allowing developers to create more robust, maintainable code.

Built to plug the holes in JavaScript, the new features allowed it to be a more strongly typed language, which appealed more to classical developers. Andres Hjlesberg, the chief architect at Microsoft, said, “In JavaScript, there’s no notion of modules or a type system.” 

According to him, TypeScript shines in supporting large codebases and a large number of people collaborating on those codebases. “JavaScript wasn’t designed to be a programming language to do any of this,” said Hjlesberg. 

A better JavaScript

It is accepted that TypeScript is better, mostly because it comes with unique features such as enums for clearer and more maintainable code, absent in JavaScript. It also offers robust type-checking and interfaces that help organise code more logically. These features are compiled into standard JavaScript, ensuring compatibility with web browsers.

The language further addresses JavaScript’s limitations by offering a systematic and predictable coding environment, making it easier to develop large-scale applications and collaborate on complex projects.

One key advantage of TypeScript is its powerful tooling, enabled by the type system, which defines what types of data can be used and how. “TypeScript is not just a language, it’s an entire set of tooling all around JavaScript,” Hejlsberg noted.

Another TypeScript’s feature is the type-inference, which is a powerful feature that allows the compiler to automatically deduce the types of variables and expressions based on their usage. Developers don’t always have to explicitly specify types, making the code more concise and readable. 

The language is compatible with existing JavaScript libraries. This means developers can gradually introduce TypeScript into their projects without having to rewrite everything from scratch. 

Apart from all its powerful features, the partnership with Angular has helped drive the numbers. Hjlesberg further pointed out, “Lots of other frameworks are using TypeScript at this point. Aurelia, Ionic, NativeScript are all, in one way or another, involved in TypeScript. The Ember framework, the Glimmer framework that was just released, is written in TypeScript.”

A JetBrains developer report of 2023 also noted that the language almost tripled over the course of six years, increasing from 12% in 2017 to 34% in 2022.  The question then had to be asked, will it eventually replace JavaScript? 

The report addresses this by saying, “Though it is rapidly catching up to JavaScript, the usage of JavaScript remains stable and high. About 92% of TypeScript developers use JavaScript, with 40% choosing it as one of their primary languages.”

The Future of TypeScript

TypeScript continues to evolve, with the team focusing on making it faster and more expressive. One challenge they are addressing is consuming unstructured responses from large language models (LLMs). 

Hejlsberg introduced TypeChat, a project that connects natural language to programming. “It’s about a pragmatic project to connect natural language to the world of programming and data structures,” said Hejlsberg.

TypeChat helps developers create structured interactions with LLMs. “We ask the models to fit the JSON into a schema and the way we tell the model about the schema is that we give it some TypeScript source code,” explained Hejlsberg. 

By providing a TypeScript schema, developers can define the expected structure of the JSON data returned by the LLM.

Another development is the “isolated declaration emit” feature, a collaboration with Bloomberg and Google. 

Daniel Rosenwasser the principal product manager at Microsoft explained, “The idea with isolated declaration emit is that each ‘public’ part of your module’s API needs to have a type annotation. When that guarantee is in place, it means that TypeScript doesn’t have to do any type-checking to generate outputs.”

Looking ahead, Rosenwasser noted, “We have heard developers want to run TypeScript with no friction — no build step if possible.” The TypeScript team is working towards this goal while seeking community feedback. Looking at the popularity the TypeScript has gained over the years, it is possible that it would overtake its parent language. 

The post Will TypeScript Wipe Out JavaScript?  appeared first on AIM.

Meta has finally dropped Llama 3, with a twist!

In its Community License Agreement conditions, the company has mentioned, “If you use the Llama Materials to create, train, fine-tune, or otherwise improve an AI model, which is distributed or made available, you shall also include “Llama 3” at the beginning of any such AI model name”, under the redistribution and use section.

Source: X

As expected, this strange clause has sparked conversations among the AI community, with many debating its need, justification, and impact.

With many startups riding on the open-source AI wave, especially in India, making it an Open-Source AI champion, it would be interesting to note how this development gets implemented.

Would we see a name change for Kissan AI’s agri LLM Dhenu, Sarvam AI’s OpenHathi, PAiGPT, and other AI models in Indic languages like Ambari, Odia Llama, Tamil Llama, all based on the Llama 2 model?

Some are just having fun online over this latest development.

Introducing Llama 3 

Meta has released Llama 3, the latest generation of its LLM. The model, available in 8B and 70B parameter versions, has been trained on over 15 trillion tokens, making it seven times larger than Llama 2’s dataset.

Llama 3 offers SOTA performance and enhanced reasoning and coding capabilities. Its training process is three times more efficient than its predecessor.

The 7B models outperforms Gemma and Mistral on all benchmarks and the 70B model outperforms Gemini Pro 1.5 and Claude 3 Sonnet. Meta is also training a model with more than 400 billion parameters.

Given its advancements, Meta’s push to include ‘Llama 3’ in the beginning of AI models that use it in their development, is a boost towards security and responsible use cases.

The post Meta Forces Developers Cite ‘Llama 3’ in their AI Development appeared first on AIM.

People in tech are sure to have come across Atlassian’s Jira, the popular project management tool. Though helpful in bug tracking and project management, it is sometimes considered as a “grumbling bug”. A simple web search on “hate jira” would throw up a flood of links and posts on X and Reddit, and a website dedicated to just hating the tool. 

On the contrary, if you veer towards the Stack Overflow Developer Survey, Jira has been ranked one of most uncomplicated and asked for asynchronous tools consistently for the past two years. Though the critics are aplenty, there are many fans too. The problem is – developers hate it, managers love it. 

Jira’s bad, but it’s best at what it does

What can be regarded as a feature by many, developers hate the fact that Jira offers a plethora of options, causing frustration. As a Reddit user said, “it’s too flexible”, making it unintuitive and a mess. 

While it may seem advantageous theoretically, opponents argue that Jira deployments often transform into cumbersome and confusing configurations filled with numerous fields, dropdown menus, and toggles, making the user experience less intuitive. Particularly when compared to newer, more efficient alternatives such as Trello, ClickUp, and Notion, Jira seems sluggish and slow. 

“Jira is just clunky as hell and enables focusing on things that aren’t the core product. It’s a high maintenance tool,” said a user on X. Just like Jira, similar thoughts have been expressed about Agile in a thread. 

“Whose bright idea was it to measure productivity by how many complexity or story points you complete? That’s like measuring a truck driver by how much diesel they use, instead of how much deliveries they make,” expressed a user on X. “You end up spending more time managing the process instead of getting the work done,” said another.

Jira as a tool was developed mostly for larger enterprises and that comes at the cost of gearing it for smaller organisations, which in the end makes it bad for developers. It is more of a tool for the decision makers in the company and such teams. 

“That means they [Jira creators] focus on satisfying the managers, and management consultants, and some class of Jira-specialised consultants, who benefit from their trade of coming into businesses to help fix their Jira setups being protected by it, not being particularly easy for organisations to use. They can get away with piss-poor experience for the devs,” explained a user on X. 

To further illustrate the user experience, another developer said that Jira’s UI is not that friendly. “I feel like it’s gaslighting me,” the user added. It is often termed as a bloated and slow software which was developed by scrum enthusiasts. 

Is Jira just a scapegoat?

Even with its problems, Jira is also admired by many when it comes to organising their workflows, but all of that comes at the cost of performance. On the other hand, several people claim that the blame on Jira is merely the blame on the management. 

A user on Hacker News said that he has never heard anyone say ‘Jira is brilliant’, but at the same time it is just a good culprit when it comes to blaming someone for the mishaps in the corporate theatre. Another use points out that the mentality of developers to sit and just work in silos and code is not something that is viable for a lot of organisations. 

“I’m a developer, and while I don’t love it, I certainly don’t hate it, and I don’t see it as worse than any other task-tracking tool,” said a user. The developers who hate Jira are the ones that have to work with bloated setups and poorly managed frameworks. This makes it more of an organisation fault than the actual tool. 

Atlassian has been accepting these feedbacks, positive and negative, and making changes to Jira. In an interview, Megan Cook, head of product for Jira Software, Agile Solutions, said that they are constantly working on improving Jira. 

Cook also proposes that Atlassian’s ecosystem will provide a competitive advantage in the age of generative AI. She emphasises that the abundance of data circulating within and around Jira can and will be leveraged to enhance the software’s functionality and utility. 

The post Why Developers Hate Jira appeared first on AIM.

“English is the hottest new programming language” – nothing irks a software developer more than this latest quip in the tech circles. But apart from English, there are several programming languages too that frustrate software engineers and developers. Some earn the ire for their syntax and style, while some for the community built around them. 

When the discussion around the most frustrating programming language went online, people chimed in. “Probably Python. Not that it’s too annoying to write, but rather it’s horribly slow and overused and overhyped for short-sighted ease of use,” said a user on X. He acknowledged that it is a good scripting language, but not for building an end product. 

While experts around the world agreed that AGI will be built with Python, it also continues to be regarded as one of the most frustrating languages by many. 

Python’s dynamic typing can lead to subtle bugs that may not be caught until runtime, and its performance can sometimes lag behind that of compiled languages like C++. Apart from people heavily discussing how dynamic typing is one of the worst things about Python, it is also the community that makes it worse for people.

As soon as someone says that Python is bad, the community calls the developer out as having “skills issues”. On the other hand, people say that any language that is not compiled, such as Ruby and Python, are the most frustrating. 

Clearing the air: Every language is good and bad

The truth about the current programming world is you can choose any language that works for you. Anything other than that is just “language bullying”. Programming languages are, after all, just tools that a programmer should learn if he needs to. If Python works for you, why go the C++ route?

Consider C++, renowned for its intricate syntax and extensive use of pointers. Many developers find navigating its complexities daunting, often encountering segmentation faults that impede progress. People, who have been using the language for decades, still don’t claim to know the language 100%. There are so many hidden intricate pitfalls.

Several people narrate their stories about adding C++ to their resume, and then when asked a specific question about it, fail to answer. This is probably because C++ is a vast language. Knowing every detail of it is not required to get the job done. Most developers choose it over Python or any other language today just for its speed.

Meanwhile, in the realm of modern languages, Rust emerges as a double-edged sword. Praised for its robust memory safety guarantees, Rust too is not without its share of frustrations. A user on X captures the sentiment perfectly: “Can’t spell frustrate without rust.”

While Rust’s strict compiler checks can be frustrating for those accustomed to more lenient languages, they ultimately lead to more robust and reliable code. Rust’s focus on safety without sacrificing performance has endeared it to developers working on system-level software and performance-critical applications. 

It is also one of Elon Musk’s favourite languages. Open-sourced Grok is also trained on Rust, but the language’s strict borrow-checker can be a stumbling block for the uninitiated.

However, C++ stands monumental when it comes to the complexities of computer programming. While its roots trace back to the venerable C language, C++ introduces a myriad of additional features, like classes, templates, and inheritance. While these features afford developers great flexibility and control, they also introduce a steep learning curve. 

The language’s syntax can be daunting, especially for those new to programming, and managing memory manually can lead to a host of bugs, including dreaded segmentation faults. Yet, for many, mastering C++ is a rite of passage – a testament to one’s dedication to the craft of software engineering.

Meanwhile, lesser popular languages like React, Go, TypeScript, Java, Erlang etc have very few users, thus the frustration of the developers using them is not really out in the open yet. 

The worst language? “The one I am working with right now”

This brings us to conclude that the most-frustrating language universally is the one most used by the developers.

According to Stack Overflow Developer Survey 2023, the most used programming language is JavaScript, and that’s something that has been consistent for the past eleven years. And undoubtedly, it is the most-hated programming language of all time. Well, some don’t even call it a programming language, but just a scripting language. 

Regardless, JavaScript, the ubiquitous language of the web, presents its own unique set of challenges. Initially designed in a hurry to add interactivity to web pages, JavaScript has evolved into a powerful, albeit, quirky programming language. Its asynchronous nature, exemplified by callbacks and promises, can confound those accustomed to synchronous programming paradigms. 

Furthermore, the ever-expanding ecosystem of JavaScript frameworks and libraries can overwhelm developers, leaving them grappling with decisions about which tools to use and when. But the truth is, it is the most-criticised language because it is the most commonly used. 

As a user put on X, “The most important thing about a programming language is how it makes you feel.” Ultimately, the key to mastering any programming language lies in perseverance, practice, and the willingness to embrace the inevitable frustrations along the way.

The post Which is the Most Frustrating Programming Language? appeared first on AIM.

Even though India’s contribution to Indic LLMs has skyrocketed in the last year, the lack of open-source pipelines for low and mid-resource languages hinders their representation in LLM training datasets. 

To address this, AI4Bharat has created IndicLLMSuite, a collection of resources for Indic LLMs covering 22 languages with 251 billion tokens and 74.8 million instruction-response pairs. Let’s take a look at some of the kit’s key resources. 

Sangraha

This includes data for pre-training data containing 251B tokens summed up over 22 languages, extracted from curated URLs, existing multilingual corpora, and large-scale translations. 

Setu

This is a Spark-based distributed pipeline customized for Indian languages for extracting content from websites, PDFs, and videos. It has in-built stages for cleaning, filtering, toxicity removal, and deduplication.

IndicAlign-Instruct

It offers a varied set of 74.7 million prompt-response pairs in 20 languages. 

These pairs are gathered using four methods, including compiling existing Instruction Fine-Tuning (IFT) datasets, translating English datasets into 14 Indian languages with an open-source translation model, generating discussions from India-centric Wikipedia articles using open-source LLMs, and setting up a crowdsourcing platform named Anudesh for prompt collection. The team has also introduced a new IFT dataset to teach language and grammar to the model, drawing from IndoWordNet, a resource-rich vocabulary. 

IndicAlign – Toxic

Finally, we have IndicAlign – Toxic, which consists of 123K pairs of toxic prompts and non-toxic responses generated using open-source English LLMs and translated to 14 Indian languages for safety alignment of Indic LLMs.

You can access the data and codes here. 

Earlier this month, Sarvam AI, along with AI4Bharat and IIT Madras, unveiled IndicVoices, a comprehensive speech dataset adhering to an inclusive diversity wishlist with fair representation of demographics, domains, languages, and applications. The IndicVoices dataset comprises  7348 hours of natural and spontaneous speech from 16237 speakers across 145 Indian districts and 22 languages. 

The post AI4Bharat Rolls Out IndicLLMSuite for Building LLMs in Indian Languages appeared first on AIM.

Google researchers recently introduced Synth2, a novel approach for training Visual-Language Models (VLMs) using synthetic image-text pairs. This new technique enhances the capabilities of LLMs and text-to-image generation, alongside addressing the limitations of manual image labelling. 

This method solves the problem of not having enough data labelled by people. It uses LLMs to generate captions and a text-to-image model to synthesise corresponding images.

Google researchers demonstrate several significant findings in terms of improved VLM performance, data efficiency, and customisation and scalability. 

For instance, the visual language model trained on their synthetic and human-annotated datasets showed marked improvement in image captioning tasks compared to a baseline trained exclusively on human-annotated data. 

“This underscores the potential of our method to augment VLM capabilities efficiently. This is highly advantageous in scenarios where data acquisition and annotation are resource-intensive,” said the researchers. 

In addition, this new approach yielded comparable performance while utilising only a fraction of human-labelled data, thereby increasing data efficiency. 

This method proves to be flexible in customising generating image datasets for specific domains. The synthetic data generation is also scalable to support large scale VLM development. 

You can find the research paper here. 

The post Google Introduces Synth^2 to Enhance the Training of Visual Language Models   appeared first on AIM.

Yann LeCun, the chief of Meta AI was recently on a podcast with Lex Fridman talking about Meta, AI, and AGI. He revealed that he recently met with an Infosys founder who is funding a project based on Llama 2. 

“He’s funding a project to fine-tune Llama 2, the open source model produced by Meta so that it speaks all 22 official languages in India. It’s very important for people in India,” said LeCun. He did not reveal which co-founder he met. 

LeCun was quite impressed by the recently released Kannada Llama. “I love this. This is why open source AI platforms will win: it’s the only way for AI to cater to highly diverse languages, cultures, values, and centers of interest.” he wrote on X, reposting Kannada Llama post.

He also spoke about Moustapha Cisse, who used to be a scientist at Meta FAIR who is also using Llama 2 to build language models in Africa, and catering it for medical use. 

LeCun is a big proponent of open source AI models and recently also posted on X about the training infrastructure of Llama 3, the upcoming AI model of Meta which is expected to be open source as well. 

Infosys was one of the first companies that had invested in OpenAI back in 2015. It donated around USD 1 billion to OpenAI, which was a non-profit back then. Infosys, along with Elon Musk, AWS, YC Research, among others, joined hands to make the USD 1 billion donation to OpenAI.

Apart from Infosys, a lot of Indian companies and startups have been leveraging Llama 2 and making Indic models such as Tamil Llama, Kannada Llama, Telugu Llama, and many more. IIT Bombay’s in work BharatGPT project is also focusing on catering to the 22 Indic languages and is pushing towards being open source.

The post Infosys Founder Funds Meta’s Llama 2 Project with 22 Indian Languages appeared first on AIM.

In the realm of modern data analytics, the ability to seamlessly translate natural language queries into actionable insights has emerged as a transformative capability. This advancement empowers users to interact with complex datasets effortlessly, extracting valuable insights and facilitating informed decision-making. However, the journey towards achieving this feat is rife with challenges, ranging from the complexity of diverse datasets to semantic sensitivity and algorithmic limitations.

Navigating the Complexity of Data

The datasets under scrutiny encompass over 500 variables across 10 distinct datasets, each varying in level of aggregation and granularity. Within this landscape, datasets with identical names may carry different meanings, while others may yield similar solutions from disparate data sources. This complexity underscores the challenge of prioritizing tables for analysis and crafting SQL queries with pinpoint accuracy. Primarily sourced from banking datasets, the information spans customer interactions, acquisition details, performance metrics, and complaint audits, underscoring the multifaceted nature of the analytical endeavor.

Addressing Semantic Sensitivity and LLM Challenges

Large Language Models (LLMs), while powerful, often grapple with semantic nuances inherent in natural language queries. Terms like “acquisition” may entail different contexts, necessitating a nuanced understanding of business and variable semantics. Moreover, variable names may vary, posing challenges in identifying the right context for analysis. To mitigate these challenges, a robust Custom  Retrieval-Augmented Generation(RAG) framework has been developed, focusing on identifying the correct business context, variables, and datasets crucial for accurate analysis. This framework leverages a Knowledge Graph to provide deeper insights into variables and datasets, ensuring precise interpretation of user queries.

Refinement with Custom RAG Framework

A pivotal aspect of the framework’s evolution lies in the development and implementation of the Custom RAG Framework. This framework emerges as a robust solution to address the intricate challenges encountered during data analysis, particularly in refining business context, variable identification, and dataset selection.

At the crux of this framework is the imperative to identify and integrate the right business context into the analytical process, since LLMs , lack the required capability to discern the subtle nuances inherent in business terminology. To bridge this gap, the framework leverages advanced algorithms to contextualize queries, ensuring a more accurate and insightful analysis.

Furthermore, the Custom RAG Framework tackles the complexity surrounding variable identification and dataset selection. In a landscape characterized by disparate datasets with varying levels of granularity and relevance, the framework streamlines the process of selecting pertinent variables and datasets. By incorporating a knowledge graph enriched with metadata, synonyms, and variable descriptions, the framework empowers the language model to navigate the intricacies of the data landscape with precision and clarity.

A notable aspect of the Custom RAG Framework lies in its adaptability to diverse use cases and industries. Whether analyzing banking data, financial institutions, or other sectors, the framework’s versatility shines through in its ability to tailor analyses to specific contexts and requirements. This adaptability ensures that the framework remains relevant and effective across a spectrum of analytical scenarios, facilitating informed decision-making and strategic insights.

Moreover, this framework serves as a catalyst for innovation, providing a platform for continuous refinement and enhancement. Through iterative feedback loops and collaborative efforts, the framework evolves to meet evolving user needs and technological advancements. This commitment to innovation underscores the framework’s long-term viability and relevance in an ever-changing analytical landscape.

In essence, the Custom RAG Framework represents a paradigm shift in data analysis, offering a holistic and refined approach to navigating complex datasets and extracting actionable insights. By integrating advanced algorithms, contextual intelligence, and domain expertise, the framework empowers users to unlock the full potential of their data assets, driving informed decision-making and strategic outcomes.

Building the Custom Scoring Algorithm and Flow of Generating SQL Queries

Central to the framework’s efficacy is the development of a custom scoring algorithm tailored to semantic search. As multiple registers failed to deliver desired performance, crafting a scoring algorithm became imperative. The algorithm functions by mining each word of the user query, identifying variables and synonyms, and computing their respective scores. Subsequently, the variables and datasets with the highest scores are selected to form the basis of the final focus for analysis.

This initial step sets the stage for generating SQL queries. The selected variables and datasets are fed into a template, enriching the context and ensuring the language model comprehends the intended analysis. Utilizing a multi-shot prompt approach, the framework guides the language model in crafting the SQL query, thereby streamlining the process of query generation.

Despite the query generation, challenges persist, such as discrepancies in date variable formats across datasets. To address such issues, an error mechanism is incorporated to rectify format inconsistencies and enhance query accuracy.  The end-to-end process culminates in the generation of SQL queries within an average timeframe of 15 to 16 seconds, facilitating prompt analysis and decision-making.

Integration with Conversational Interface and Enhancing User Experience

Transitioning from the educated end-to-end framework to a conversational interface marks the next phase of the analytical journey. Preparatory work, including the creation of a knowledge graph and vector stores, sets the stage for seamless interaction with end consumers via the interface.

Upon user query initiation, the framework springs into action, leveraging the enriched knowledge graph to augment prompts and guide SQL query generation. The generated query interfaces with the data warehouse, retrieving relevant datasets and executing the query to generate tabular outputs. This output, also comprising insights, charts, and textual analysis, is relayed back to the conversational interface, empowering end users with actionable insights.

Democratizing Data Analysis and Future Enhancements

The ultimate goal of the framework is to democratize data analysis, enabling users to derive insights without delving into intricate coding processes. Early results indicate significant time savings and enhanced productivity, with insights generated up to three to four times faster and manual work reduced by 60 to 70%.

To further refine the framework, continuous improvement efforts are underway. These include fine-tuning the custom scoring algorithm, expanding metadata to enhance fluidity in query interpretation, and implementing user suggestions to address data integration gaps. By iteratively refining the framework based on user feedback and technological advancements, the aim is to elevate the efficacy and usability of the tool, ultimately empowering users with unparalleled data analysis capabilities.

In conclusion, the custom RAG framework represents a significant advancement in addressing the challenges of analyzing complex datasets. By leveraging custom scoring algorithms and integrating business context into algorithmic analysis, the framework streamlines SQL query generation from English language queries. Its end-to-end flow, from knowledge graph creation to UI presentation, facilitates swift and efficient insight generation. The framework’s emphasis on democratizing insight generation and its potential for further refinement promise to drive data-driven decision-making across sectors.

The post Democratize data analysis and insights generation through the seamless translation of Natural Language into SQL queries appeared first on AIM.

Leading education company upGrad is harnessing AI to translate popular learning materials like bootcamps and certifications into Indian languages. 

Presently, more than 40% of upGrad’s learners in India hail from Tier 2 cities and beyond, where local languages are prevalent. In response to this demographic insight, the company aims to translate its offerings into Hindi, Tamil, Telugu, Kannada, and Bengali in the initial phase.

To enhance accessibility and effectiveness, upGrad will integrate a blend of pedagogy and human interventions. Local language-speaking counsellors and buddies will be deployed to guide learners through the educational journey. 

“Considering that only 7 to 8% of graduates in India are deemed employable, our objective is to substantially elevate this figure by providing millions with some initial support or springboard to get closer to their career aspirations,” said upGrad president Asheesh Sharma.

Sharma highlighted the company’s investment in AI, stating that they have been using AI in operations for a while and are now integrating it further into the curriculum and teaching methods. This strategic move aligns with the company’s growth and transformative goals for the fiscal year 2025.

The focus in Phase 1 will cover domains like engineering, data science, AI, cloud computing, DevOps, UI/UX, and agile project management, with Phase 2 expanding into areas such as cybersecurity, blockchain, product management, business analysis, and ITIL, along with foreign languages like Spanish and Chinese.

With over five lakh enrolled learners, the demand for upGrad’s courses has been notable in key cities such as Bengaluru, New Delhi, Mumbai, Hyderabad, Odisha, Pune, Chennai, and Kolkata. To address inquiries effectively, the company will leverage an in-house tech tool, aligning with the Government of India’s initiative to connect culturally and foster cognitive development among the youth.

The post upGrad is Using AI to Translate Learning Materials into Indian Languages appeared first on AIM.