Hello there! If you're diving into this article, I'm guessing you have a thing for Ruby on Rails (RoR). If not, then perhaps curiosity has led you here—welcome either way! 😄

RoR, known for its elegant syntax, vibrant community, and the almost magical metaprogramming capabilities, truly makes coding an enjoyable experience. The syntactic sugar and the "magic" one-liners can often feel like a programming ballet. However, it's crucial to remember that every dance comes with its own set of moves that, if not executed correctly, can lead to performance pitfalls

Well I have some bad news, the syntactic sugar comes at a cost of performance and bad practices, so in this brief article we will go over some of the most common mistakes we do in our everyday coding and how to fix them.

1-The Misconception Around pluck

While it's often suggested that pluck should be avoided in favor of select to prevent database calls, the actual advice should be more nuanced. pluck directly converts a database query result into an array of values for specified columns, which can be more efficient than loading ActiveRecord objects when you only need one or a few columns. The real optimization is understanding when to use pluck versus select. Use pluck for retrieving values of a single column directly without the overhead of building ActiveRecord objects. Use select when you need to utilize the returned objects in Rails, especially if you're chaining more queries onto the result.

# Use when you need an array of values from one column
User.pluck(:id)

# Use when you need ActiveRecord objects to chain more queries
User.select(:id, :name)

ActiveRecord queries are lazily loaded. This means the query is not executed until the data is actually needed. This can be both an advantage and a disadvantage, depending on how the query is used in the view.

Consider a model method that's used in a controller and then viewed in the view. If this method returns an ActiveRecord relation, the database query is only executed when the data is actually accessed in the view:

In the model :

class User < ApplicationRecord
  def published_articles
    articles.where(published: true).select(:title, :content) 
    # This does not execute the query
  end

  def bad_published_articles
    articles.where(published: true).pluck(:title, :content) 
    # This executes the query
  end
end

In the controller :

class UserController < ApplicationController
  def show
    @user = User.find(params[:id])
    @articles = @user.published_articles 
    # This does not execute the query
  end
end

In the view :

<% @articles.each do |article| %> <!-- The query executes here -->
  <%= article.title %>
<% end %>

The pluck method is often misunderstood. While it's true that indiscriminate use of pluck over select can lead to performance issues, understanding when to use each can significantly optimize your queries. pluck shines when you need an array of values from a single column without the overhead of ActiveRecord objects. Conversely, select is your go-to when those ActiveRecord objects are needed for further manipulation or when chaining more queries.

Pro Tip: Utilize pluck when working with APIs or background jobs where ActiveRecord object instantiation is unnecessary and could lead to increased memory usage.

2-Using exists? Instead of any? or count

When you want to check if any records exist that meet certain criteria, it's more efficient to use exists? instead of any? or counting records. exists? will stop scanning as soon as it finds the first record that matches the condition, which is more efficient than loading multiple records into memory.

Pro tip : Use exists? for feature toggles or conditional logic in views where you need to check the presence of associated records without loading them

# Less efficient
User.where(active: true).any?
User.where(active: true).count > 0

# More efficient
User.exists?(active: true)

3- Understanding count vs. size

count:

• The count method performs an SQL COUNT query against the database.
• Every time you call count, it executes a new SQL query, which can be less efficient, especially on large tables or complex associations.

size:

• The size method is more intelligent and adaptive than count.
• If the association has been loaded, size will return the result without hitting the database by counting the elements in the loaded array.
• If the association has not been loaded, size will perform an SQL COUNT query, similar to count.
• This makes size more efficient in scenarios where the records are already loaded into memory because it avoids unnecessary database queries.

Pro tip: Default to size for a balanced approach to efficiency and accuracy, especially when working with associations that might already be loaded.

Wrapping Up

Optimizing RoR applications involves a blend of understanding ActiveRecord's intricacies and applying best practices judiciously. While syntactic sugar makes RoR delightful, it's the mindful use of its features that ensures both productivity and performance.

Stay tuned for more insights, and remember, the goal is not just to write code but to write code that scales gracefully and efficiently. Happy coding, and let's keep making magic with Ruby 💎!

Hello Tech GEEKS 🤓
Today, I'm gonna walk you through a crucial topic for every software engineer. As software engineers, every day we are trying to cope with newer technologies, and most importantly, the newer versions of our daily used technologies.

In this guide, I'm gonna walk you through upgrading Angular and Ionic versions of your existing app. We will take a deep dive into the steps for the upgrade, tips for a faster upgrading process, issues, and how I managed to solve them.

🤝🏻Introduction:

We needed to upgrade our Salesman app's Angular and Ionic versions to cope with the latest versions. The Salesman app was a kind of legacy app that used Angular v12 and Ionic v6.

We all know that the latest version of Angular is v17 ,and the latest version of Ionic is v7. After the upgrade, I succeeded in upgrading it to Angular v16 and Ionic v7. So, let's start our journey together with some tips and steps for the upgrade.

💡Tips and Steps

Tips for the upgrade

• When upgrading your app to newer versions, you have to upgrade it version by version. You shouldn’t upgrade from v12 to v16 directly; instead, you should upgrade from v12 to v13, from v13 to v14, and so on, until you reach v16.
• After you upgrade your app to the newer version, fix any issues related to the upgrade.
• Replace any deprecated feature or package with a new one.
• You should check all the dependencies and fix any conflicts you face.
• Test your app with the newer version.

Steps for Upgrading Angular

• Go to the Angular Update Guide: https://update.angular.io/
• Select your current app version and the newer one (select from and to versions).
• Choose Application Complexity: “Hint: You can keep it basic initially, and after you finish the steps, go to the next complexity and take a look.”
• Every upgrade has a minimum version of dependencies, such as nodejs, typescript, Zone.js. You should check them first and upgrade them if needed before upgrading the Angular version.
• Follow all the steps in this guide.

Steps for Upgrading Ionic

• Go to the Ionic Update Guide:
  https://ionicframework.com/docs/updating/7-0
• Every version may need a minimum version of dependencies such as RxJS, angular-server, angular-toolkit
• Follow all the steps in this guide.
• After the upgrade, you may need to replace some ionic HTML tags or CSS attributes; it depends on the Ionic version.

Tips and Tricks

• Upgrade only one version at a time.
• Separate each upgrade into a new branch: When you upgrade Angular to version 13, create a branch: upgrade-angular-to-v13. After you fix the conflicts, upgrade dependencies, and test the app, Create a new branch for upgrading Angular to version 14: upgrade-angular-to-v14 and do the upgrade, and so on.
  
  Every new upgrade will be in a new branch. This makes you detect the issues faster, and it is easier to rollback the upgrade.
• Run npm outdated to list all the outdated packages in your package.json
• Run npm outdated [package-name] to detect specific package versions.
• Run npm install <package-name>@<version> to install a specific version from npm
• Run ionic info to print project, system, and environment information (ionic, capacitor, and cordova versions).
• Run ng version to print versions information like (Angular, Node, npm, typescript, RxJS).
• Learn how to read the dependencies conflict errors: https://stackoverflow.com/questions/76039613/how-do-i-read-npm-dependency-conflict-errors
• When you find conflicts between a dependency and its peer dependencies, you have to check the npm and GitHub repo for each to view latest update , versions , releases notes, compatibility table if exists, releases , and issues

• Here’s an example of how to fix compatibility and versions using releases on the GitHub repo:
  • Go to this GitHub repo for angular/toolkit: https://github.com/ionic-team/angular-toolkit/releases
  • You will see that every release supports a specific version of Angular, so you need to pick the supported version of your own.
  • in this example: I upgraded @ionic/angular-toolkit@9.0.0 and @ionic/cordova-builders@9.0.0 when I upgraded Angular to version 15.

• Another example is solving ng2-file-upload version conflicts.
  • Go to this GitHub repo: https://github.com/valor-software/ng2-file-upload/issues/1255
  • In this issue, they suggest upgrading ng2-file-upload to version 5 because this version supports Angular 16.

• Another example is solving ionic-native/core outdated
  • go to this npm link: https://www.npmjs.com/package/@ionic-native/core
  • When you click the GitHub link, it will redirect you to awesome-cordova-plugin instead, and after some research, I found that awesome-cordova-plugin is the new version of ionic-native/core , but it doesn’t have a datepicker. Check out this link: https://ionic.io/blog/a-new-chapter-for-ionic-native
  • When checking the ionic-native/core, I realized that the last version was published 2 years ago, which means it is outdated. So I searched for a replacement for this package because we use the datepickers from it, and I found capacitor-community/date-picker which does the same work.

• Another example is solving the cordova-plugin-ionic JIT error
  • Go to this npm link: https://www.npmjs.com/package/cordova-plugin-ionic
  • I found that the last version was published a year ago, which means it doesn’t support Angular 16. and we use Deploy from cordova-plugin-ionic for the live-updates (hot deploy).
  • cordova-plugin-ionic doesn't support AOT compilation (AHEAD OF TIME); it supports only JIT( JUST IN TIME).

• Here’s the compatibility table for Ionic v7: https://github.com/ionic-team/ionic-framework/blob/main/BREAKING.md#version-7x
• Here’s the compatibility table for ngx-translate/core and ngx-translate/http-loader : https://github.com/ngx-translate/core#installation

After we finished the tips and steps section ✅,

We learned how to solve conflicts and issues, how to organize the upgrading process, and some useful commands.

We're ready to dive into updating our app 🏃🏻.

💻 Upgrading angular and Ionic 🚀

Upgrading Angular from v12 to v13

When upgrading Angular to v13, I faced 3 issues: will discuss them and their solutions:

Issue 1 : "@angular-devkit/build-angular" and "@angular/compiler-cli" conflicts

• Error Explanation: Your project is trying to use version 13.3.11 of @angular-devkit/build-angular but it has a peer dependency on @angular/compiler-cli version 13.0.0 or 13.3.0-rc.0.
• Solution:
  • remove node_modules and package-lock.json to reset all the dependencies and their peers: rm -rf node_modules package-lock.json
  • then run npm install to install all dependencies again.

Issue 2 : Mismatch between packages

• Error Explanation: type mismatch between two versions of the @sentry/types package. One version is being used by your application (@sentry/types/types/integration) and another version is being used by the sentry-cordova package (sentry-cordova/node_modules/@sentry/types/dist/integration).
• Solution:
  • To fix this issue, you should ensure that both your application and the sentry-cordova package are using the same version of the @sentry/types package. You can do this by updating your package.json file to specify the correct version of @sentry/types, and then running npm install to update your dependencies.
  • upgrade @sentry/tracing and sentry/cordova to be compatible with each other.
  • npm i @sentry/tracing@7.34.0
  • npm i sentry-cordova@1.2.0

Issue 3 : browser compitability

• Error Explanation: When testing the app in the emulator, a blank screen appears instead of the app. and this is because of browser compatibility.
• Solution:
  • To fix this issue, you should specify Chrome version
  • Open .browserslistrc file and copy these versions supported by angular
  • Chrome >=60 Firefox >=63 Edge >=79 Safari >=13 iOS >=13

Upgrading Angular from v13 to v14

• When upgrading Angular to v14, I didn’t face any issues, but there is a new approach introduced in Angular 14 regarding forms, which is typed forms.
• After the migration from 13 to 14, all the import forms changed to untyped/forms in all the components that used forms to not break any existing forms.
• I removed this auto migration and kept them as forms, not untyped/forms and tested our forms, and they worked well.
• Typed Form explanation link: https://angular.io/guide/typed-forms

Upgrading Angular from v14 to v15

When upgrading Angular to v15 related to dependencies issues, I didn’t face any issues but did some upgrades:

• Some dependencies didn’t update automatically, so I upgraded them manually by running npm i @angular-devkit/schematics@15 @angular-devkit/core@15 @schematics/angular@15
• Upgraded Ionic 6 to the latest version by running npm install @ionic/angular@6
• Upgraded capacitor 4 to the latest version by running npm i -D @capacitor/cli@latest-4 and all its native packages by running npm i @capacitor/android@4 @capacitor/app@4 @capacitor/core@4 @capacitor/haptics@4 @capacitor/keyboard@4 @capacitor/splash-screen@4 @capacitor/status-bar@4
• Upgraded awesome-cordova-plugins 6 to the latest version by running npm i awesome-cordova-plugins@6
• Upgraded ngx-translate/core and ngx-translate/http-loader
  • ngx-translate/core 14.x and ngx-translate/http-loader 7.x support Angular 15.
  • ngx-translate/core 15.x and ngx-translate/http-loader 8.x support Angular 16+.
  • npm i ngx-translate/core@14 http-loader@7
  • compatibility table: https://github.com/ngx-translate/core#installation
• Upgraded ionic/angular-toolkit and ionic/cordova-builders by running npm i @ionic/angular-toolkit@9 @ionic/cordova-builders@9
  • release notes guide: https://github.com/ionic-team/angular-toolkit/releases
• Upgraded RxJS 6 to the latest version by running npm i rxjs@6
• Run this command npm outdate [package-name] to check whether the package is outdated or not.
  • When checking all Cordova plugins, I found that these needed to be updated: cordova@^12 cordova-browser@7 cordova-plugin-android-permissions@1.1.5 cordova-plugin-geolocation@5 cordova-plugin-statusbar@4
  • npm i cordova@^12
  • npm i cordova-browser@7 cordova-plugin-android-permissions@1.1.5 cordova-plugin-geolocation@5 cordova-plugin-statusbar@4
• Upgraded capacitor to version 5 by following this guide: https://capacitorjs.com/docs/updating/5-0

Upgrading Angular from v15 to v16

When upgrading Angular to v16, I upgraded some dependencies and faced some issues:

• Upgraded zone.js needed to be version 0.13.x or later npm i zones@0.13
• Upgraded @schematics/angular@16 npm i @schematics/angular@16

Issue 1: ng2-file-upload 4 didn’t support angular 16

• solution: upgraded ng2-file-upload to v5 that supports v16 npm i ng2-file-upload@5

issue 2: ionic/storage doesn’t support angular 16, needed to use ionic/storage-angular instead

• solution:
  • Installed ionic/storage-angular npm i ionic/storage-angular
  • Applied the new implementation from the docs: https://github.com/ionic-team/ionic-storage

issue 3: found that cordova-plugin-ionic doesn't support AOT compilation (AHEAD OF TIME), it supports only JIT (JUST IN TIME) and we use Deploy for hot deploys from cordova-plugin-ionic

• solution: Implemented live-updates from capacitor docs: https://ionic.io/docs/appflow/deploy/setup/capacitor-sdk

Issue 4 (enhancement): The canActivate interface is deprecated in Angular 16, so they suggest implementing it as a normal method instead of a module.

code snapshot before the migration

code snapshot after the migration

Upgrading Ionic from v6 to v7

When upgrading Ionic to v7, angular-toolkit needed to be at version 9, but we upgraded it before, so we didn’t need this step. I found some issues, which will be discussed below.

Issue 1: RxJS version conflicts

• Error Explanation: RxJS needed to upgrade to version 7.5.0 but we couldn't do this because the ionic-native/core and datepicker we use in the app need RxJS 6.5. ionc-native/core last release was 2 years ago, so it is outdated.
• Solution:
  • I upgraded RxJS anyway and replaced ionc-native/core with capacitor-community/date-picker
  • GitHub link: https://github.com/capacitor-community/date-picker
  • npm link: https://www.npmjs.com/package/@capacitor-community/date-picker

Issue 2: Browser Compatibility

• Error Explanation: When testing the app in the emulator, a blank screen appears instead of the app. and this is because of browser compatibility.
• Solution:
  • To fix this issue, you should specify the Chrome version as the documentation says: https://ionicframework.com/docs/updating/7-0
  • Open .browserslistrc file and copy these versions supported by Angular.
  • Chrome >=79 ChromeAndroid >=79 Firefox >=70 Edge >=79 Safari >=14 iOS >=14

After fixing these issues, I followed the Update your code section of this guide: https://ionicframework.com/docs/updating/7-0 to check whether I needed to change any code or not. This code may be HTML tags or CSS attributes related to Ionic components.

📚References

• Angular Update Guide: https://update.angular.io/
• Ionic Update Guide: https://ionicframework.com/docs/updating/7-0
• Capacitor Update Guide: https://capacitorjs.com/docs/updating/5-0
• Live update capacitor SDK:
  https://ionic.io/docs/appflow/deploy/setup/capacitor-sdk

Our APIs were suffering from slow response time and this was affecting the user experience on our apps. One of those is an important API which is responsible for fetching products on sale. The problem was resulting from N+1 queries. In this post, we’ll explain what N+1 queries are, how to detect them in your rails application and how to debug in order to eliminate them.

What is N+1 Query

The N+1 query problem is one of the common performance anti-patterns which happens when the data access layer executes N additional SQL statements to fetch the same data that could have been retrieved when executing the primary SQL query [1].

API Performance Before Fix

We can see that the API in the screenshot below is suffering a lot, as reported by NewRelic, a performance monitoring tool, with very slow average response time and a low Apdex score (i.e: a ratio value of the number of satisfied and tolerating requests to the total requests made) [2].

Below is the segment breakdown for the API response time through 2 days before deploying the fix, we can see that it doesn’t go below 500ms.

How To Detect N+1 Queries

Sentry

Sentry is an error tracking and performance monitoring platform which can be integrated with your rails application, I will leave resources below explaining how to integrate Sentry into your rails application [3, 4].

Sentry reported that our API is suffering from an N+1 query issue, and pointed out the repeated query as shown in the screenshot below:

NewRelic

Click on APM & Services from the left sidebar, then select the suspect API. Click on Transactions under the Monitor tab then select the API endpoint which appears to be on the top time consuming transactions. This will open a tab that contains transaction details about the selected API endpoint..

By viewing the break down table for the database transactions in the API endpoint, it’s obvious that the pointed out queries are having high Avg. Calls/Transaction which is a symptom for N+1 queries.

Debugging N+1 Queries

Bullet Gem

To detect where exactly the N+1 query is happening in your code, bullet gem comes to the rescue [4]. It’s a gem that watches your queries while you develop your application and notifies you when you should add eager loading (i.e: eager loading is the process by which a query for one type of entity also loads related entities as part of the query, so that we don't need to execute a separate query for related entities), when you're using eager loading that isn't necessary. Integration with bullet gem is rather simple, the docs go through it very well so I’m not going to cover that in this article.

By enabling bullet gem and allowing rails_logger option, bullet will create a log file for you that records the stack traces for queries that suffer from N+1 query. After that, hitting the concerned API endpoint while enabling bullet gem, it logged the N+1 query with the stack trace as shown below.

2022-10-31 10:57:25[WARN] user: root
GET /api/v1/supplier_offer_products?supplier_id=9&branch_id=13&area_id=300&app_version=3.3.1
USE eager loading detected
  CoreEngine::SupplierOfferProduct => [:product_supplier]
  Add to your query: .includes([:product_supplier])
Call stack
  /app/engines/core-engine/app/models/core_engine/concerns/can_override_product_prices.rb:87:in `gross_selling_price'
  /app/engines/core-engine/app/services/core_engine/products/product_pricing_service.rb:118:in `each'
  /app/engines/core-engine/app/services/core_engine/products/product_pricing_service.rb:118:in `min_by'
  /app/engines/core-engine/app/services/core_engine/products/product_pricing_service.rb:118:in `block in load_and_merge_available_product_prices'
  /app/engines/core-engine/app/services/core_engine/products/product_pricing_service.rb:118:in `transform_values!'
  /app/engines/core-engine/app/services/core_engine/products/product_pricing_service.rb:118:in `load_and_merge_available_product_prices'
  /app/engines/core-engine/app/services/core_engine/products/product_pricing_service.rb:38:in `load_available_prices'
  /app/engines/core-engine/app/services/core_engine/products/available_products_and_prices_service.rb:41:in `load_available_prices'
  /app/app/controllers/api/v1/supplier_offer_products_controller.rb:40:in `get_offer_products'
  /app/app/controllers/api/v1/supplier_offer_products_controller.rb:20:in `index'
  /app/app/controllers/application_controller.rb:43:in `set_time_zone'

The log here shows the code trace that led to the N+1 query and which relation we need to include while fetching the resources .includes([:product_supplier]). We can see that there’s a loop as identified in the code trace transform_values!, which is a map function that loops on the resources and performs a hit to the database to retrieve a ProductSupplier record.

@product_supplier_prices_hash.transform_values! { |priceable_list| priceable_list.min_by(&:gross_selling_price) }

This line is sorting the prices based on gross_selling_price and the resources in  @product_supplier_prices_hashdoesn’t include ProductSupplier relation and  gross_selling_price is defined as follows:

def gross_selling_price
	product_supplier.gross_selling_price_for_price selling_price
end

So it performs a query to the database to retrieve the record and get gross_selling_price value from it. This happens on each product in the hash. If we check the method that retrieves SupplierOfferProduct records, we will find that it is missing .includes([:product_supplier]) to eagerly load the ProductSupplier records and have them ready in memory instead of querying the database on each SupplierOfferProduct.

def load_offer_products
    CoreEngine::SupplierOfferProduct
    .is_published
    .of_product_supplier(@product_supplier_ids)
    .available_from_supplier_or_campaigns(
       @route_ids.join(','), @retailer.supplier_class_ids.join(','), @retailer.active_campaign_ids
    )     
end

Including the ProductSupplier relation by adding .includes([:product_supplier])  to the method eliminated the N+1 queries on this relation.

Avoid Adding Logic Querying Database in Serializer

For the other two queries, the problem was that we were returning the supplier info in the API response as shown below:

render  json: all_offer_products, current_retailer: @current_retailer,
        distribution_routes_mapping: @distribution_routes_mapping,
        include: [product supplier brand category]

It’s very important to be careful with what you return in your API response, as that will affect two things:

• response payload size
• response time

The supplier attribute is added to each SupplierOfferProduct returned but it wasn’t actually required  in the API response as it gets called in the supplier page to list the products which are on sale by the supplier, so the supplier info is already fetched by another API call. However, let’s see what caused the N+1 queries.

Checking the SupplierSerializer there’s an attribute returned in the serialized object which is defined as follows:

attribute :cashback_info do
     ActiveModelSerializers::SerializableResource.new(
     CoreEngine::Suppliers::CashbackPercentageService.new(current_user.retailer, object).call,
         each_serializer: CashbackSerializer,
         root: 'cashback'
     )
end

What happens here is that we’re getting the cashback_info which is determined by the commission rate and the campaigns the supplier is subscribed to, those two relations weren’t eagerly loaded. So, for each SupplierOfferProduct we’re adding to it a serialized object of the Supplier hence the N+1 queries.

Conclusion

After eagerly loading ProductSupplier records and removing supplier attribute from the API response, the N+1 queries were completely eliminated and the API performance has increased significantly.

Checking the API performance on NewRelic before and after the fix we can see that the avg. response time decreased by 70% and Apdex score increased by 22%.

Looking at the segment breakdown for the response time below, we can see the drop in the response time after deploying the fix on 28th of October.

Takeaways

• It is very useful to have performance monitoring tools for your applications that will help you detect performance issues. By investigating and fixing these issues, your application performance can improve significantly.
• Make sure to eagerly load records needed and keep them in memory to avoid hitting the database and adding unnecessary latency to your API requests.
• Make sure to avoid adding unnecessary data to your API response.
• Make sure to avoid as much as you can adding logic that queries the database in your serializer.

Resources

• [1]: https://dev.to/junko911/rails-n-1-queries-and-eager-loading-10eh
• [2]: https://docs.newrelic.com/docs/apm/new-relic-apm/apdex/apdex-measure-user-satisfaction/
• [3]: https://docs.sentry.io/platforms/ruby/guides/rails/
• [4]: https://docs.sentry.io/platforms/ruby/guides/rails/performance/
  

One of our mobile apps used by tens of thousands of retailers in Egypt is suffering in performance, mainly in the SupplierProducts page which lists the products sold by a specific supplier. Most of the complaints indicate that the app performance degrades while navigating through multiple suppliers. And eventually, the app crashes. Usually, this is a symptom for memory leaks [1].

Debugging Process

Under Chrome Developer tools, recording a heap allocation timeline and using the performance monitor provide strong insights about what is happening according to memory allocations.

Heap Allocation Timeline

Before starting to record heap allocation, I made sure to take a note of the current heap size and it was 19.5MB.

After visiting multiple supplier products pages, I found that none of the visited pages are garbage collected (see the figure below). This means that there are some references to those objects which prevents the GC from collecting the allocated memory.

We can see that the retained size (i.e: the amount of memory that will be free once the object is garbage collected) for SupplierProductsPage and SupplierProductCardComponent which is the product cards inside the page are almost 1.72MB.

We can also see that the heap size jumped from 19.5MB to 57.6MB (38.1MB up, almost a 200% increase).

Objects taking a significant size and bloating the memory are detached elements, which are elements removed from the DOM tree but are still referenced somewhere so they don’t get garbage collected.

We can see in the figure below by checking the retained size column for each detached element group that the retained size by each group is huge.

Performance Monitor

The performance monitor gives you a real-time view of various aspects of run-time performance, it gives you graphs showing the total number of DOM nodes created, JS Heap size, and more.

Recording the DOM nodes creation and JS heap size while visiting multiple suppliers indicates that the total number of nodes created and JS heap size graphs are taking the shape of a ladder which means that they keep growing only, almost taking the shape of a straight line going up.

Retainer Tree

Retainer Tree represents the objects that have reference to the selected object.

Checking the retainer tree for each of the SupplierProductsPage objects indicates there’s a common reference between them which is el in Swiper which is a Detached HTMElement.

Swiper comes from ion-slides which is an ionic element and internally works using SwiperJS.

Digging into ion-slides

Doing some search about ion-slides, I found out that it has been deprecated and Ionic suggests migrating to SwiperJS and using it directly for Angular [2].

By checking the retainer tree for the swiper element, it turns out there is an event listener that is still alive which holds a reference to the element.

Digging into the code for swiper in @ionic/core, I found two event listeners; one on orientationChange, and the other on window resize in order to resize the swiper element in both cases.

The problem was that the destroy method was never invoked to remove the listeners, as I tried adding two console.log one in init method and the other in destroy method and fount that destroy was not invoked on leaving the page, and that was causing a memory leak [3].

var Resize = {
  name: 'resize',
  create() {
   // some logic
  },
  on: {
    init() {
      const swiper = this;
      // Emit resize
      win.addEventListener('resize', swiper.resize.resizeHandler);

      // Emit orientationchange
      win.addEventListener('orientationchange', swiper.resize.orientationChangeHandler);
    },
    destroy() {
      const swiper = this;
      win.removeEventListener('resize', swiper.resize.resizeHandler);
      win.removeEventListener('orientationchange', swiper.resize.orientationChangeHandler);
    },
  },
};

By commenting the line for adding the two event listeners and retry the same scenario found that there were no memory leaks.

Solution

The solution to migrate to SwiperJS as advised and install the latest version because @ionic/core was using v5.4.1, released on May 2020, and is so outdated.

Adding console.log inside destroy method of resize.js file in the newer SwiperJS, I found that the destroy method was invoked and the event listeners were removed and no memory leaks occurred[4].

Results

Before starting recording the heap size was 18.9MB.

After migrating to SwiperJS and visiting the same number of suppliers while recording the heap allocation timeline, all the objects were garbage collected and there was no trace for SupplierProductsPage or SupplierProductCardComponent objects.

We can see that the heap size is 23.1MB (4.2MB increase).

We can also see that there are no traces of the huge amount of detached elements.

Also by checking the performance monitor for DOM Nodes and JS Heap size we can see that the timeline is taking the shape of the healthy saw tooth wave which indicates that memory gets allocated and then deallocated.

Compare that to the corresponding chart prior to the fix and you can see a 400% improvement in Dom nodes count, and a much healthier JS heap size chart!

Conclusions

• Checking for memory leaks should be done on a regular basis because any new change, even if not by your engineers, can cause memory leaks..
• You should always keep up to date with the recommendations of the tools and frameworks you’re using.
• Event listeners should always be removed once not needed to avoid memory leaks [5].

References

• [1]: https://developer.chrome.com/docs/devtools/memory-problems/#overview
• [2]: https://ionicframework.com/docs/api/slides#migration
• [3]: https://github.com/ionic-team/ionic-framework/blob/7e5c03ae4096ada8bc7388975a83e51aab55cce5/core/src/components/slides/swiper/swiper.bundle.js#L4376
• [4]:https://github.com/nolimits4web/swiper/blob/0567641fbe4bdbea54b5de0bcc2c937dfb52e088/src/core/modules/resize/resize.js#L62
• [5]: https://www.ditdot.hr/en/causes-of-memory-leaks-in-javascript-and-how-to-avoid-them#event-listeners
  

In this article, we will walk you through how to detect memory leaks in your web application and how to investigate the reasons behind those memory leaks and solve them using a real-world example. We will have a journey together to get from snapshot 1 to snapshot 2.

Note: In this article, we are using Ionic v5 and Angular v9.1.12.

Memory leaks are one of the most challenging optimization issues to tackle, they can have multiple sources and usually, you will need to find the pattern that causes a memory leak. That's why it's crucial to learn how to measure memory performance and get insights from memory allocation timelines.

What Is A Memory Leak?

In computer science, a memory leak is a type of resource leak that occurs when a computer program incorrectly manages memory allocations in a way that memory that is no longer needed is not released ¹. In short, a memory leak can be defined as some memory allocated that is not required anymore but not released, i.e,  not garbage collected.

Most modern programming languages manage memory automatically. The memory lifecycle usually consists of three steps:
1. Allocate memory needed
2. Read from and write to allocated memory
3. Release the memory allocated as soon as it's not needed anymore

Angular does a great job at memory management but in some cases that results from bad practices in implementation leads to memory leaks and that has a very bad impact on the user experience.

Debugging Process

Because the performance of our application degrades with time throughout the user session this is a strong indication of the existence of some memory leaks. To study that we needed to monitor the memory use in real time.

Step 1: Monitor Memory Allocation In Real-Time

Chrome DevTools has two very helpful memory debugging tools for monitor memory allocation in real-time:

1. Performance Monitor
2. Record Memory Allocation Timeline Heap Snapshot

Let's start with the performance monitor.

Performance monitor

If we open the performance monitor it starts recording the heap size throughout the session. We can see in the graph below that it has the shape of a sawtooth, the heap size is getting bigger over time and that indicates that there are some memory allocations that have not been garbage collected.

Now we need to get our hands on the objects that are not garbage collected. To find that we will take a memory allocation timeline heap snapshot

Recording Memory Allocation Timeline Heap Snapshot

To record a memory allocation timeline make sure to refresh the page and click on the collect garbage icon, to clean the memory from any previous session leftover objects.

Note: in an Angular web application make sure to set the optimization flag in angular.json file to false in order to be able to search for the components by their names.

In the memory allocation timeline below those blue bars represent new memory allocations, and grey bars represent garbage-collected memory. We can see that on each store's page we enter that memory gets allocated but when we leave the page the memory is not freed and that means we have some memory leaks.

Before we dive into finding the reason for the memory leaks, let's talk briefly about how garbage collection works in JavaScript. In Javascript, resources get garbage collected when there are no references to it. That means that even if an object is not needed anymore but still referenced and accessible from another place in our application, it will not be garbage collected. This issue can't be handled by the garbage collector, it's up to the developer to clean up the references to the objects.

Step 2: Investigating The Reasons For Memory Leaks

After stopping the recording and you click on an object you can filter the objects by their names. On this page, we have two components SupplierProductsPage and SupplierProductCard which are rendered inside the former page as a child component.

If we filter the objects by name we will find that we have 3x SupplierProductsPage objects and 60x SupplierProductCard objects as we show 20x products on entering the page. Also, we can see some other objects created by Angular Ivy which have a very large Retained Size -- i.e the size of memory that is freed once the object itself is deleted along with its dependent objects that were made unreachable from GC roots.

If you click on an object you will find a window get shown called object's retaining tree. This tree shows us the objects that have reference to the selected object. After going through the object's retaining tree of  SupplierProductsPage object and SupplierProductCard object we found that there are subscriptions to RxJS Observables that are still alive and causing the memory leaks.

RxJS is a library for composing asynchronous and event-based programs by using observable sequences. It combines the Observer pattern with the Iterator pattern and functional programming with collections to fill the need for an ideal way of managing sequences of events ².

In RxJS it is very important to remember to unsubscribe from an observable for two reasons:

1. A subscription has a reference to the component they are created in. So, when the component gets destroyed it will not be garbage collected and that will result in a memory leak.
2. The code will continue to execute in the background even if the component gets destroyed, and that can result in unpredictable errors and side effects.

The solution to this issue is quite simple yet can go unnoticed very easily. We need to hook into ngOnDestroy component lifecycle to clean up our subscriptions in both components before the component gets destroyed.

Step 3: Applying Solutions

Upgrading To Angular 10

We found that in Angular 9, Ivy is leaking memory and it doesn't deallocate memory of destroyed components correctly. In this issue you can find people complaining about it. So we tried upgrading to Angular 10 to see if this issue is resolved in the newer version and we found that it did get resolved.

In the heap snapshot below we can see that the LEmbeddedView_ and LComponentView_ don't exist anymore.

Result Of Unsubscribing From Observables

The subscriptions have been cleaned up, however, after taking a new record for the memory allocation timeline the object's did not get garbage collected.

To find the reason why the objects didn't get garbage collected event after unsubscribing it took some time because I found that the problem was not in our implementation but it's a bug in an Ionic component which is ion-img, this component lazily loads images.

Describing The Bug In ion-img Component

We came across this issue, in the comments we found that someone has reported that ion-img component implemented by Ionic didn't clean up its IntersectionObserver references properly. If you checkout the code for the implementation, you will find that the IntersectionObserver doesn't get disconnected on the component destroy.

To confirm that we tried to use a native img tag and see if the components are garbage collected. We recorded a new memory allocation timeline to check the number of objects for each component.

In the recording below, we can see that each time we visit a store's page the garbage collector runs and the previous memory allocations get garbage collected, and that proves that the component is the problem.

We can see here that the number of SupplierProductsPage object is only x1 and the number of SupplierProductCard object is only x20 which is the number of product cards we render on entering the page.

Surprisingly, when we tried to use loading="lazy" attribute on the native img tag, the issue still persists and the components don't get garbage collected ³.

Implementing Lazy Loading

Since we can't eagerly load images on all pages to avoid performance issues, we decided to implement lazy loading ourselves to make sure we disconnect from the IntersectionObserver on component destroy.

To avoid reinventing the wheel, we tried to look if someone has created a directive for images lazy loading and we came across this amazing implementation by Ben Nadel. The implementation needed to get updated because it was implemented for Angular 5 and we use Angular 10.

After Implementing lazy loading as a directive added it to the native img tag using IntersectionObserverAPI we had the same result as using the native img tag, and that's because we made sure in the implementation that we disconnect from the IntersectionObserver when the component gets destroyed.

Results

After applying the above solutions:

• Upgrading to Angular 10
• Unsubscribing from observables on components destroy
• Removing ion-img component from all components and implement lazy loading

To compare the heap snapshots we tried to perform the same actions on each session, we found that the heap size in production mode has decreased significantly from 18.9mb to 11.6mb. Also, we can see that the performance monitor graph has improved and it shows that there are multiple drops in heap size throughout the sessions.