TVLK5: The Late Intro

By now you have to hear about this project called TVLK5. This post is an attempt to explain briefly what it is, how does it differ from how we used to develop products, what's changed since initial introduction and what's next.

TVLK5 is an umbrella project to improve web development in Traveloka customer facing site, specifically how we author our code. It's technically our 2nd big change (but who's counting right), with the first one being Blocks and TvPage.

It's easy to call this a framework but we don't actually hide any backing implementation this time. It consists of express+nextjs boilerplate and few modules that you can integrate with. If you need to modify server middleware, you still working with an express app. The same can be said with nextjs, you follow their convention, with the entry inside pages directory. Want to do something fancy with nextjs? Just head to their documentation and it should work the same.

This is the main difference with our legacy system where you're kinda sucked in to how the framework was built, with routes being defined in specific file not even using express convention, middleware replaced with Filters and so on.

To understand how much is changed compared to our previous framework you can read some of these comparison below

Hello World

In TVLK5, to create new page you only need to touch single file (pages/route.tsx following next.js convention), compared to 5+ files you need to modify in previous framework (up to 2 routes, server module, client module, component, and webpack entry point).

If you need access to some properties, you might also need to configure your redux store. Ouch

Bonus point: you're not blocked by web-infra anymore for code review.

Build speed

In CI, entire TVLK5 workflow (JS build, docker+push, and performance tests) is faster than JS build of our monolith app. With faster feedback loop means you wait less time for your PR to be merged just because there's minor adjustment that you have to make.

Locally, you get the benefit of HMR where you can see your change in an instant. In our testing, adding log statements to the code will take > 20 seconds in previous framework, while the new one only took ~2 seconds. That's 10x improvement for a single feedback loop.

Content Resource & friends

In previous framework, fetching translation and any resource data are tied strongly to react-diode implementation which makes it difficult to optimize because CR/IR/IS have unique needs. In TVLK5 the syntax is simpler and still familiar, yet it have benefits like:

• Code split friendly. We leverage React.Suspense to make sure that you can place any resource query anywhere in the tree and it will be fetched only if your component renders it. You don't need to add children manually, it will automatically inferred based on request.

// Before:
Diode.createContainer(Component, {
  children: [Child],
  queries: Diode.createQuery(ContentResourceQuery, {
    GeneralLayout: {
      logIn: null,
    },
  }),
});

// After:
useContentResource({
  GeneralLayout: {
    logIn: '',
  },
});

• Dead fields elimination. When you fetch image resource / image sliders, there's a lot of fields returned by backend API where most of them aren't even used. In TVLK5, those fields will be automatically removed if you don't use it, similar to how GraphQL works.

// Before:
Diode.createContainer(Component, {
  children: [Child],
  queries: Diode.createQuery(ImageSliderQuery, {
    // All fields will be hydrated in JSON even if it's not used
    PaymentPartnersDark: {},
  }),
});

// After:
useImageSlider({
  PaymentPartnersDark: {
    // only link will be hydrated
    link: '',
  },
});

Mobile redirection

This is common usage yet previously there's a few boilerplate when adding this functionality. In TVLK5, this can be as simple as single line change

// Before:
function pageRedirect(req, res) {
  const url = req.router.url('PAGE_ROUTE_KEY');
  return res.redirect(url);
}

// After:
Page.mobileRedirection = true;

There's an advanced redirection logic that you can do, but most of the time you don't need that anyway. Why bother writing to much code?

Confusing Lifecycle

Previously we have requestReceived, beforeFetch and afterFetch. There's no clear way regarding what should be done inside those lifecycle, you only know what happen.

In TVLK5, by leveraging next.js we have getInitialProps that you can do to parse request data and fetch from backend API, or routeProtection that you can use to block access to specific route.

// Before
class Page extends TvPage {
  requestReceived(req, res) {
    // wait is there req.user?
    if (req.user) {
      res.status(403);
      return ResponseHandler.endPageLifecycle();
    }
  }
  beforeFetch(req, res) {
    super.beforeFetch(req, res);
    req.props.key = 'value';
  }
  afterFetch(req, res) {
    // ????
    super.afterFetch(req, res);
  }
}

// After
Page.getInitialProps = async (ctx, appCtx) => {
  return { key: 'value' };
};

Page.routeProtection = ctx => {
  if (ctx.user.loggedIn) {
    return { status: 200 };
  }

  return { status: 403 };
};

Parallel server-side API call

Using Filter in previous framework means each filter is executed serially, this means that any API call happen inside different filter will be called sequentially.

In TVLK5, we optimize to fetch as much data as possible to reduce server latency. We even provide you with Server Timing to see how each server-side call performs.

Production debugging

Due to modular approach, it's now feasible to test production build locally without breaking your laptop. You can build and run production build using either staging or production API.

# Before:
cd packages/nodejs-web
NODE_ENV=production pnpm build
## if it doesn't crash, then
NODE_ENV=production node lib/index.js

# After:
pnpm --filter @traveloka/webxpe-desktop build
pnpm start

Client-side routing

Previously, to use client-side routing, you have to change a lot of things, starting from changing your base page, your route registration, your component render method, and more. This is assuming you don't want to code split because it's a different case altogether (thanks to Diode). It's not a pleasant experience.

In TVLK5, client-side routing get first class support, which means it works by default without you having to reconfigure anything. In next.js every page is different bundle so it's code-split by default.

// Before:
// duplicate path declaration with registration?
<Link to="/path/:key/another" />
// or "server-side" navigation
window.location.assign(
  // Don't forget to add this in RouteResourceGroups
  router.url('SOME_KEY_THAT_YOU_NEED_TO_FIND_IN_REGISTRATION_FILE')
)
// and another ceremonies


// After:
<Link route={ROUTE_PATH} params={{ key: 'value' }} />
// or programmatic client-side navigation
const route = useLocalizedRouter();
const [href, as] = route(ROUTE_PATH, {
  params: {
    key: 'value',
  }
});
Router.push(href, as)

Those are non-exhaustive list of what's new in TVLK5. If you're curious, read our getting started docs.

Migration Requirement

If you want to migrate your services to use TVLK5 these are few things you should know:

• TVLK5 only officially supports @traveloka/web-components as component primitive. This is probably one of the major blocker / where you'll need to spend most effort in. We'd suggest you refactor your component to use web-components before performing any migration.
• There's no more dependency to nodejs-web. Each services can only depend to @traveloka/core, @traveloka/app-desktop, @traveloka/app-mobile, and other micro packages (such as @traveloka/user or @traveloka/merchandising). You can extract your utility modules to your own package, and make nodejs-web depend to that package, but not the other way around
• The new services will only handle language-country routing, e.g: /id-id prefix. This means your service have to be compliant with URL standardization effort. In the future nodejs-web will only be used as redirection for naked locale and legacy /en prefix.

Updates

At this time of writing there's 3 product in production using TVLK5: Experience, DLP, and Explore. Thanks to experience team for becoming pilot for this adoption 🎉!

Since XPE team released their servies to production, we have made some improvement to the framework:

• There's now 3 experimental features that you can try to improve your page performance: SSR cache, progressive hydration, and static subtree
• We have completed migration of shared desktop and mobile components (header, footer, sidebar) available in @traveloka/app-desktop and @traveloka/app-mobile. If your page uses this component and you already use web-components for all your components, you'll have easier time migrating your services.

We also received contribution from various engineers in feature parity, bug fixes and other enhancements:

• Thanks to Agal, we now have better OTP integration and external login capability
• Thanks to Yusei we now have frictionless setup using Yeoman generator.
• Thanks to Ferdinand, Raibima, Billy, Paula for bug fixes and DX improvements.
• Thanks to Dzulham and Ferdinand we have external tracking manager setup with automatic segment pageview tracking.
• Thanks to Ryan, we have easy performance tracking with mock recorder. This means adding performance test is as easy as visiting your page in dev and modify 1 JSON file.
• Thanks to Abi and Asendia we have migrated to Jenkins with the ability to do partial build. This means in average new TVLK5 services can finish building in under 10 minutes, including performance tests (the production build itself should finish in about a minute).

What’s Next?

While most of the groundwork has already finished, there's more that we haven't got to it yet:

• Adoption of nextjs advanced features (dynamic route, module/nomodule, rewrites). Using dynamic route allow us to create route with params without touching express code, while module/nomodule will improve our page performance in newer browsers due to different transpilation method
• Staging & Release DX improvement. This is only slightly related to TVLK5 itself but it's an issue. Currently there's so many thing to touch when you want to create new services and release it to production. We're planning to simplify this process so you can migrate your screen 1 by 1.
• Dev DX improvement (especially logging and hanging process). We noticed there's a strange bug where some process are not closed even after you stop the main proxy. This can cause issues in performarnce and battery life. If possible, we also like to remove nohoist limitation when you setup your packages. This could hopefully improve installation time
• Docs completion, most of the docs are already completed, but the advanced topics sections (especially Architecture) are still missing. We plan to finish the remaining docs in H2.

If you have any suggestion on what to improve in the framework, don't hesitate to contact us in frontend-club channel.