hyva_checkout performance

We noticed some performance issues with hyva_checkout. On our website we have a lot of grouped/bundle/configurable products with lots of options. Customer normally can have more than 10-15 items in the shopping cart. We looked in blackfire and found out that HyvaCheckoutLayoutLoadBefore observer takes a lot of time on each page to load current customer quote, is it really necessary??

Here are some screenshots from blackfire Category pages: Homepage:

Product detail page:

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information