// archive

Writings

5 entries

Rust, ownership, borrowing and lifetime

It has been three months since I began my journey with Rust. I started by watching a course video on freeCodeCamp, then I continued by building some simple projects and testing out some frameworks to hone my skills in Rust.Well, yes, Rust has some new concepts that we will talk about in this article and a unique learning curve, which makes it more interesting. However, it is still the same as the other programming languages. You do not have to be scared to learn it.Now, let’s move to some new concepts that were introduced by Rust. OwnershipLet’s say we have thislet num = 1; let word = String::from("Hello"); let num2 = num1; let word2 = word; println!("{}", num); println!("{}", word);If we run this code, Rust will complain in line 8, the word doesn’t own the value anymore. Why is this happening? In line 5, we pass the ownership of the variable to word2.To illustrate it, imagine you have a brother. At your brother’s birthday, you gave him your favourite toy. This process is equivalent to changing ownership in Rust, like how word passes it to word2. Hence, you won’t be able to play with that toy anymore unless you borrow it, which leads to another concept in Rust.After reading the code, you might have a question. Why can the variable num still have a value? For the detailed answer, we might have to understand how Rust stores its data in memory, but let's keep that for another day. To make it short, the process that happens in line 4 is copying the data. Unlike variable word that have type String, num has a type of integer, so copying data is cheap. BorrowingFrom the story, we know that you can borrow the toy from your brother. So does the variable in Rust. There is a concept called borrowing. Let’s take a look at the code below:let word = String::from("Hello"); let word2 = &word; println("{}, {}", word, word2);We can see in line 2 that instead of giving the ownership, we pass the reference to word2. That’s called borrowing in Rust. You can lend your toy without giving it away forever.But take note. There are certain rules in Rust related to borrowing:You can have many variables reading the value at the same time.But only one variable can modify it exclusively if they want to change it. LifetimeNow, imagine if there’s a timer. Rust wants to know, “How long will that toy be around?”That’s called a lifetime.fn playground() -> &String { let toy = String::from("Hello"); &toy }To illustrate, if you are in a playground that borrows you a toy. You can play with the toy in the playground. However, you cannot take it home.In Rust, the variable is gone when the function ends. So you cannot give a reference anymore since the data has already disappeared from memory.

Rust

Is Redis the right call?

I've been using Redis for quite a while, and I've been treating it as magic. Database is slow? Add Redis.API is slow? Add Redis.Are sessions slow? Add Redis.Is everything slow? Have you tried Redis?It is like we're using Redis as a performance insurance, but that's not the case.It is indeed that Redis is fast, but it is expensive since it stores data in memory. Well, let's talk about several cases where we shouldn't rely on Redis. Your queries are not optimizedInstead of immediately implementing Redis, why don't we start by optimizing our database queries?Redis can boost application performance, but it won't solve inefficient queries. It may only hide the underlying problem.For example, you might discover that your application is suffering from the N+1 query problem. Before adding Redis, we should first ask: can we reduce the number of queries, add the right indexes, or optimize how we fetch the data?Fix the query first. Then, if the database is still a bottleneck, consider Redis. You have complex queriesAn example of this is a listing page in e-commerce. You are building a page that requires you to query multiple tables, join them, and add where statements.  Then, suddenly you found out that the queries are so slow. You started to think about adding Redis as a cache. That might work, but you can quickly end up dealing with complicated cache keys, invalidation logic, and difficult debugging, especially when your listing page has many filters and facets.In this scenario, the better solution might not be Redis. You may need to optimize your database queries or, if the read workload has genuinely outgrown your current database, consider a read-optimized database with CQRS. I want to tell you about this one in this section, but you can find a lot on the internet about the usage of OpenSearch, Algolia, or TypeSense. You use Redis for heavy computationYes, Redis can support some computation using LUA scripts or Redis functions, but it doesn't mean you can put a heavy operation in Redis. Why is this bad? Redis is single-threaded when executing a command operation. Therefore, if you have a heavy operation, it will block other operations, causing a bottleneck. In fact, you should move that operation to another service that runs asynchronously, so it won't block your other operations. I am writing this because I hate Redis. In fact, I love using Redis in my projects since Redis is a great tool and almost helps me with every performance problem. But sometimes we are not using the tool correctly.  

Redis

What Happens When Rust Compiles a Generic Function?

Like other programming languages, Rust allows us to work with generics and abstractions.In Rust, there are two ways we can handle types that implement a particular trait.The first is when we use a generic type as a function parameter or return value.The second is when we need to work with a concrete type via a single interface, such as a Vec.Although both serve the same purpose, Rust handles them differently.The first approach uses static dispatch, which requires the compiler to know the concrete type at compile time. Meanwhile, the second approach uses dynamic dispatch. We use trait objects to handle this one since the compiler doesn't know the concrete type and size at compile time. However, we won't go deep on this one.Let's get back to the first approach. So, what does it mean for the compiler to know the concrete type at compile time, and what does the compiler actually do?Let's take a look at this example:fn addition(a: T, b: T) -> T where T: Add, { a + b } fn main() { println!("{}", addition(10, 20)); println!("{}", addition(10.5, 20.5)); } When the compiler sees addition(10, 20), it can determine that both arguments are i32. Therefore, for this call, the compiler knows that T = i32. At this point, the compiler knows the exact types used for each call.After the compiler gathers this information, it will continue with a process called monomorphization.The idea is simple. For each function that uses generic types, Rust will generate a specialized version for the concrete types that are actually used.Therefore, for the previous example, conceptually, the compiler will produce something similar to:fn addition_i32(a: i32, b: i32) -> i32 { a + b } fn addition_f64(a: f64, b: f64) -> f64 { a + b } fn main() { println!("{}", addition_i32(10, 20)); println!("{}", addition_f64(10.5, 20.5)); } The advantage of using this is that there is no runtime lookup cost, but it will make the compiled code bigger.

Rust

Overview of AI Agent Evaluation

When developing a software application, it is common practice to have a set of tests and evaluations. This practice helps us to ensure that it works as expected and is reliable in production.Then, what about AI agents? Do we need to evaluate them?Of course we do, but it is more challenging than traditional applications.So, how do we evaluate them?Well, that's a tough question. But before we jump into how we evaluate AI agents, let's first understand how traditional applications are evaluated. Evaluation in traditional software applicationIn a traditional software application, its behavior is mostly deterministic, which means that the output is predictable when we can control the variables, such as input, application state, and application environment.For example, let's say we develop a BMI calculator that takes weight and height to calculate the body mass index, rounded to one decimal place.If we evaluate it using a height of 155 cm and a weight of 40 kg, our application should consistently return a BMI of 16.6 regardless of how many iterations we have.If the application returns a different result, such as 16.5, when using the same inputs and conditions, the test should fail. Evaluating AI AgentsWhen evaluating AI agents, we should understand their nature and behavior. Unlike traditional software applications, AI agents may produce different outputs even when given the same input. This is partly due to their reliance on Large Language Models (LLMs), which generate responses based on probability rather than following strictly deterministic rules.For example, let's say we develop an AI agent to respond to user greetings. If we input "Hi" into the agent, we might expect it to respond with "Hi, how are you?". However, that's not always the case. The next time we provide the same input, we might get a response like "Hello! How can I help you today?". However, this behavior is not entirely random. There are several variables that can influence how an LLM generates its response, such as the LLM model, session context, or temperature (but we won't cover it in this article). This variability makes evaluating AI agents more challenging than evaluating traditional software applications.Now that we understand how AI agents behave, let's move on to how we evaluate them.There are two types of evaluation we can perform:\ DeterministicIn deterministic evaluation, we check whether an AI agent's response meets specific criteria. For instance, we can assess whether the response contains specific information or avoids specific words.Here are several examples:Let's use our previous BMI calculator and transform it into an AI agent. Then, we will have a system instruction as below:"You are a helpful assistant. Your task is to calculate Body Mass Index using this formula to one decimal place: BMI = weight(kg) / height(m)^2"Then, when the user uses this prompt:"I have a height of 155 cm and a weight of 40 kg, calculate my BMI."Our AI agent might respond with this:"You have a BMI of 16.6"or“16.6”In this case, we can evaluate our AI agent by expecting "16.6" to be in its response by using regex or a simple contains function. Furthermore, for more robust evaluation, we could extract the numerical result and compare it against the expected value.Let's say we build a polite AI agent and we don't want it to respond with bad words, such as fuck, what the hell, and many more. Therefore, when the user inputs this message:“Fuck You”We can evaluate it to ensure its response doesn't contain the word "fuck" by using regex or a simple not in statement.Aside from the examples above, we can also use deterministic evaluation to verify whether an AI agent is working as expected. This includes assessing its harness or execution behavior, such as the number of tool calls it makes or which tools it uses to complete a specific task. IndeterministicDeterministic evaluation is useful, but it is not enough to evaluate an AI agent since it can be difficult to assess certain aspects of its response using predefined rules. To address this limitation, we can use another LLM as a judge to evaluate our AI agent's responses based on specific criteria. There are several metrics we can use, such as faithfulness, correctness, relevance, and completeness.For example, in our previous BMI agent, the agent might respond like this:"Your body mass index is 25 (obesity type I). To improve your condition, I suggest you exercise regularly."However, we don't want the agent to provide additional information aside from the BMI value. Therefore, we improve our system prompt as follows:"You are a helpful assistant. Your task is to calculate Body Mass Index using this formula to one decimal place: BMI = weight(kg) / height(m)^2 Your response must only contain the BMI value without any additional information."After updating the system prompt, we can use an LLM as a judge to evaluate whether the agent follows this instruction and whether its response meets our evaluation criteria.

AI AgentEvaluation

Exploring Loop Fission vs. Loop Fusion in Rust

When I was playing LeetCode a long time ago, I found someone who posted an interesting discussion. He gave us a tip to separate a single loop to process an algorithm into separated loops to make it faster. At first, I thought he was joking since it logically doesn't make sense. However, when I dug more into it, I found that the technique of splitting one loop into multiple loops is called loop fission. It is the counterpart of loop fusion, where we combine multiple calculations into one single loop whenever it is possible. Surprisingly, loop fission can sometimes perform better than loop fusion.I wanted to understand why, so I conducted three simple experiments using Rust.Experiment SetupTo perform this experiment, I used Rust and measured execution time for these two approaches:Loop Fusion: Performs all calculations inside a single loop.Loop Fission: Separates or distributes the calculations into multiple loops. Both approaches use the same experiment variables, such as input data, and perform the same calculations. To verify the result, a checksum will be checked at the end of each test.Each experiment uses six input sizes, starting from 10,000 to 100 million elements. The test will be run seven times and use the median execution time.Case A: Simple CalculationsIn this case, we will perform two independent simple calculations on a u64 vector:Calculate the sum of consecutive indices.Multiply each element by its previous element.Results (median of seven runs):Surprisingly, I did not expect the results to be quite close, while fission is faster in five out of six tests. I initially thought fusion would be faster because it processes the data in a single pass.Nevertheless, this experiment shows that reducing the number of iterations does not always improve performance.Case B: Memory Pressure (Random Access)For the second experiment, we will see how memory access affects performance.There will be three u64 vectors: one input vector and two output vectors. Both calculations access the input using pseudo-random indices.Results (median of seven runs):This time, the results are opposite from the previous one. Fusion performs better in most cases, especially for larger input sizes, with fission becoming approximately 13–28% slower.To investigate why, I examined the 10-million-element test using Cachegrid:From the data, we can see fission produces more instructions executed and almost twice as many cache misses compared to fusion. This might be one of the reasons why fission is slower.One possible explanation is temporal locality. When fusion performs calculations within its loop iteration, it accesses the same input locations close together in time. This gives the processor a better opportunity to reuse recently accessed cache lines. Meanwhile, fission performs the second calculation in a separate pass, making those cache lines less likely to remain available.This experiment demonstrates that loop fusion can be beneficial when calculations reuse the same data.Case C: Complex Calculations (36-Term Polynomial)For the final experiment, let's do complex calculations.Each iteration calculates two independent 36-term polynomials using f64 values. Fusion performs both calculations in the same loop, while fission separates them into two loops.Results (median of seven runs):This is interesting. Fission is consistently faster by around 9-13% across all sizes. To understand why, I asked the coding agent to examine the generated assembly and used Valgrind to compare instruction counts and cache behavior.The Valgrind test used a smaller dataset of 100,000 elements, which fits comfortably within the CPU's last-level cache. This helps isolate CPU execution from memory pressure.The results showed:The fused version executes approximately 1.4 million more instructions, while both versions have virtually identical cache misses.The generated assembly showed that the fused loop requires additional instructions to load constant values. Separating the calculations allows the compiler to generate more efficient instructions.This suggests that the performance difference is not caused by memory access, but by the additional instruction overhead in the fused version.Final Thoughts The biggest lesson for me is that fewer loops do not automatically mean better performance. Memory access, cache behavior, register usage, and compiler optimizations can all influence the final result.These experiments are only a small exploration using specific workloads and hardware. They don't establish that one technique is always better than the other.My takeaway: Don't optimize code based on how it looks. Write clear code first, measure its performance, and investigate the bottleneck before deciding whether loop fusion or loop fission makes more sense. Understanding trade-offs is important.Source code: https://github.com/aodihis/loop-research

PerformaceFissionFusionLoop