(Optimization) Lazy evaluation : Postpone until the result is actually needed

In software engineering, we are sometimes taught to be prepared. We initialize variables, construct objects, and compute values early in a scope to ensure they are ready for use. This is known as “eager evaluation.”

However, we must be careful not to cause performance degradation due to unnecessary operations. Even copying a std::string, can significantly degrade performance when executed frequently in hot paths.

Lazy evaluation is the strategy of deferring a computation until its result is strictly required. A specific subset of this concept is lazy creation, which postpones creating instance(s) until the instance is actually required.

---

Example 1

/* Before change */
std::string NameHandler::get_name_with_new_pattern(std::string name) {
    for (const auto& [pattern, new_pattern] : pattern2new_pattern_) {
        if (name.compare(0, pattern.size(), pattern)) {
            name.replace(0, pattern.size(), new_pattern);
            return name;
        }
    }
    return "";
}

In the code above, the function accepts the name parameter by value (copy) in anticipation of the string::replace operation.

However, if the if condition is not met and the function returns an empty string, that copy is never used.

To optimize this, we can apply lazy-creation by deferring the creation of the string copy until the if condition is satisfied and the object is strictly required. The revised code is shown below:

/* After change */
std::string NameHandler::get_name_with_new_pattern(const std::string& name) { // Use const-reference 
    for (const auto& [pattern, new_pattern] : pattern2new_pattern_) {
        if (name.compare(0, pattern.size(), pattern)) {
            std::string new_name = name; // Lazy-creation
            new_name.replace(0, pattern.size(), new_pattern);
            return new_name;
        }
    }
    return "";
}

---

Example 2

Trace is a logging utility class that stores the name of the target function in a member variable.

/* Before change */
// Trace.h (Logger class)

class Trace {
public:
    Trace(const std::string& name); // Constructor
    ~Trace(); // Destructor
    void debug (const std::string& msg); // Print debug message

    static bool active_ = false; // Determine whether to print message
private:
    std::string func_name_; // Name of function where instance is created
};

// *** Inline function definitions ***************
inline Trace::Trace(const std::string& name) : func_name_(name) {
    if (active_) {
        std::cout << "Enter function : " << func_name_ << std::endl;
    }
}

inline Trace::~Trace() { 
    if (active_) {
        std::cout << "Exit function : " << func_name_ << std::endl;
    }
}

inline void debug (const std::string& msg) {
    if (active_) {
        std::cout << msg << std::endl;
    }
}

A usage example is shown below:

int my_function(int x) {
    std::string name = "my_function";
    Trace trace(name);
    ...
}

Analyzing this example reveals two distinct inefficiencies of Trace class:

1. The caller must construct a std::string object just to invoke the constructor.

2. The member variable func_name_ is unconditionally constructed, regardless of whether it is actually used (which is determined by Trace::active_)

We can apply the following optimizations to address these issues.

1. Change the constructor parameter type from const std::string& to const char*.

2. Prevent the unconditional construction of the std::string member variable (Lazy-creation).

/* After change */
// Trace.h (Logger class)

#include <memory>

class Trace {
public:
    Trace(const char* name); // Use pointer instead of reference
    ~Trace();
    void debug (const std::string& msg);

    static bool active_ = false;
private:
    std::unique_ptr<std::string> func_name_ = nullptr; // Use pointer for lazy-creation
};

// *** Inline function definitions *************
inline Trace::Trace(const char* name) {
    if (active_) {
        func_name.reset(new std::string(name)); // Lazy-creation
    }
}