Speculation Rule

Speculation Rule is designed to improve performance for future navigation. It targets document URLs rather than specific resource files, and makes sense for multi page applications rather than single page applications.

Developers can take advantage of speculation rules to prerender page, which can greatly reduce the wait time for users. This method is particularly beneficial for pages like flight and accommodation searches.

However, while using speculation rules can improve the user experience, it also puts more strain on our infrastructure. Therefore, it's essential for developers to keep an eye on the performance and load of related services when implementing these rules.

Enablement of speculation rule

Turn on feature control

Before using the speculation rule, first go to feature control and enable the speculation rule for your purpose.

For example, to enable the flight search page speculation rule, the flight team has set up this feature control rule:

(Enabled) featureId:speculation-rule enabled:true
(Text) featureId:speculation-rule key:flightSearch value:ENABLE

After you set up the rule, please extend the SpecFeatureControlTypes in packages/core/speculation-rule/types.ts.

The team can then query the feature control result by the following code.

// Extend Feature Control Page Types
//packages/core/speculation-rule/types.ts
export type SpecFeatureControlTypes = 'flightSearch' | 'newType1' | 'newType2';

// Query Feature Control
import { useEnableSpeculationRule } from '@traveloka/core';

const fcSpeculationRule = useEnableSpeculationRule('flightSearch');

if (fcSpeculationRule.enabled) {
  // do something
}

Check browser compatibility

Feature control is currently supported on about 68% of devices. While Chrome has good support, it is not available on Safari or Firefox. Developers can use isHTMLSupportSpeculationRule to check browser compatibility. It is useful when you want to adopt another optimization method when speculation rule page is not available.

import { isHTMLSupportSpeculationRule } from '@traveloka/core';

if (!isHTMLSupportSpeculationRule) {
  return prefetchSearchAPI();
}

Inject Speculation Rule

Enable speculation rule is simple and straightforward. Below is an example.

actionType is required. It is used for tracking the hit rate of different speculation page types. After you select a actionType for your page, please extend the type in packages/core/speculation-rule/types.ts. It is recommended to keep your feature control key type and action type the same.

// Extend actionType
//packages/core/speculation-rule/types.ts
const PrerenderActionTypes = ['flightSearch', 'newType1', 'newType2'] as const;

// Inject speculation rule page
import { InjectSpeculationRule } from '@traveloka/core';

const fcSpeculationRule = useEnableSpeculationRule('yourType');

if (fcSpeculationRule.enabled) {
  return (
    <InjectSpeculationRule
      actionType="yourType"
      rules={{
        prerender: {
          url: searchLink,
          target_hint: '_self',
        },
      }}
    />
  );
}

Defer actions

By default, all metrics (internal trackings, external trackings, page view trackings, datadog) are deferred until the page is activated.

If you want to defer other actions until activation, use pageActivated. For instance, to measure users' actual page load time, subtract getActivationStart from performance.now(). getActivationStart gives the time between page load and page activation.

import { specRuleHelper } from '@traveloka/core';

const { pageActivated, getActivationStart } = specRuleHelper
pageActivated().then(() => {
  console.log('user waiting time', Math.max(0, performance.now() - ))
})

Inspection

Speculation rule pages are rendered in the background. To inspect a prerendered page:

Add the speculation rule script
Open DevTools -> Application -> Background Services -> Speclative loads.
Click Speculations -> Speculative Loading Attempt and then Inspect

This opens the prerendered page panel, where you can view console logs and network activity.

For more details, see: debugging speculation rules

Local Dev Environment

WebSocket does not work well on speculation rule pages. Since webpack HMR relies on WebSocket during development, speculation rule pages cannot be developed directly.

Instead, developers should first build the app and serve it locally. Below is a list of steps to set up local development for speculation rule.

Change api endpoint:

We will run the web product service and the webapx service together. When the webapx service runs on its own, it uses port 8080.

// core/api/callAPIClientSide.ts
- let targetAPIURL = '/api' + apiPath + qs;
+ let targetAPIURL = 'http://localhost:8080' + '/api' + apiPath + qs;

Add CORS support to the local webapx service:

The Access-Control-Allow-Headers list may grow over time. If you encounter a CORS error due to an unsupported header, extend the list as needed.

// proxies/api-proxy/main.go
// Add Cors middleware as follows:
r.Use(
    gin.Recovery(),
    otelgin.Middleware(
        serviceName,
        otelgin.WithSpanNameFormatter(func(c *gin.Context) string {
            return fmt.Sprintf("PROXY %s", c.Request.URL.Path)
        }),
        otelgin.WithFilter(func(r *http.Request) bool {
            // Ignore healthcheck trace
            return r.URL.Path != "/healthcheck"
        }),
    ),
    logger.Middleware(),
    // ----------------------- Add the following --------------------------
    func(c *gin.Context) {
        c.Writer.Header().Set("Access-Control-Allow-Origin", "http://localhost:2900")
        c.Writer.Header().Set("Access-Control-Allow-Methods", "GET, POST, PUT, PATCH, DELETE, OPTIONS")
			  c.Writer.Header().Set("Access-Control-Allow-Headers", "Origin, Content-Type, Accept, Authorization, x-route-prefix, x-did, x-domain, x-client-interface, t-a-v, tv-country, tv-currency, tv-language")
        c.Writer.Header().Set("Access-Control-Allow-Credentials", "true")
        if c.Request.Method == "OPTIONS" {
            c.AbortWithStatus(204)
            return
        }
        c.Next()
    },
)

mock feature control values(optional)

//packages/core/speculation-rule/useEnableSpeculationRule.ts

return useMemo(() => {
  return {
    // enabled: featureEnabled,
    enabled: true,
  };
}, [enabled]);

turn on datadog log(Optional)

Do this only if you want to test datadog defer in dev enviornment.

Firstly, copy and paste datadog rum config from staging environment to development environment:

// copy from packages/core/config/staging.js
// copy datadogRUMConfig

// paste it to packages/core/config/development.js
datadogRUMConfig: {
  applicationId: '7d4ac777-b8ed-469a-882d-f302edeba889',
  clientToken: 'pub4b83d41b849ea5327216c53705ac8a63',
  sessionSampleRate: 100,
  sessionReplaySampleRate: 100,
  traceSampleRate: 100,
  defaultPrivacyLevel: 'mask-user-input',
},

Secondly, set logHandlers to datadogLogs.logger

// packages/core/logger/index.client.js
// const logHandlers = [
//   getPublicRuntimeConfig().environment !== 'development' && datadogLogs.logger,
//   getPublicRuntimeConfig().environment !== 'production' && console,
// ].filter(Boolean);

const logHandlers = [datadogLogs.logger].filter(Boolean);

Build service locally

pnpm --filter @traveloka/webfpr-desktop run build

Start service locally

pnpm --filter @traveloka/webfpr-desktop run start

Start webapx service locally

pnpm --filter @traveloka/webapx start

After running the commands above, you should be able to see an instant load of the page you specified.

Metrics

Hit Rate

The prerender hit rate is calculated as (total_activated_prerender) / (total_injected_prerender) and measures the effectiveness of a speculation rule injection.

total_injected_prerender counts each InjectSpeculationRule component only once, even if the rule updates multiple times. This ensures the hit rate reflects actual bandwidth usage, since updated content is mostly served from browser cache.

When a speculation rule is injected into the DOM, InjectSpeculationRule sends a Datadog metric speculation.rule with the tag phase: inject_prerender.

When a prerendered speculation rule page is activated, it sends a Datadog metric speculation.rule with the tag phase: activate_prerender.

Trackings

Both internal and external tracking are deferred until the page is activated. To fire a track event even if the page isn’t activated, use the track function by calling:

import { useTracker } from '@traveloka/core';

const track = useTracker({
  deferTilActivated: false,
});

Server Observability

All API calls from a speculation rule prerendered page include the header Sec-Purpose: prefetch;prerender. When proxied through webapx, this header is preserved, allowing the backend to identify calls from a prerendered page.

Note: Once the HTML page is activated, subsequent requests will no longer include the Sec-Purpose header.

Performance measurement

Sending metrics to measure page load performance is recommended. Here’s an example from the flight search page.

Also, include the has_referrer field. Chrome may enable speculation rule pages when the user types in the address bar. has_referrer helps distinguish Chrome predictor prerenders from our speculation rule enablement.

import { specRuleHelper } from '@traveloka/core';

const { pageActivated, isSpecPage, getActivitionStart } = specRuleHelper;
const activateStart = getActivitionStart();

pageActivated().then(() => {
  logger.info('speculation_flight_search', {
    name: 'speculation_flight_search',
    is_spec_page: isSpecRulePage(),
    trip_type: tripType,
    route_type: routeType,
    interface: interfaceType,
    country: country,
    has_referrer: Boolean(document.referrer),
    time: {
      page_load_to_render: page_load_to_render,
      page_load_toactivate: activateStart,
      activate_to_render: Math.max(0, page_load_to_render - activateStart),
    },
  });
});

API

InjectSpeculationRule

Inject speculation rule page. If the browser does not support speculation rule, it will return null. Your can place this code anywhere in your component tree. InjectSpeculationRule in rendered inside next/head, so it is inserted into the head dom node.

Updating or removing InjectSpeculationRule will discard the previous speculation rule page.

import { InjectSpeculationRule } from '@traveloka/core';

<InjectSpeculationRule
  // fill in your intention here.
  // please define actionType in 'core/speculation-rule/types'
  actionType="flightSearch"
  rules={{
    prerender: [
      {
        // The page you wish to prerender
        url: url,
        // optional, default to '_self'
        target_hint: '_self',
      },
    ],
  }}
/>;

actionType

A string specifies the type of prerender page to inject. This value is used in Datadog to track hit rates for different prerender types.

Initially, actionType only includes flightSearch. To add more types, update PrerenderActionType in packages/core/speculation-rule/types.ts.

url

The URL to prerender, which can be a full or relative path. Cross-site prerendering is not supported.

target_hint(Optional)

A string specifying where the prerendered content will be activated.

Possible values: '_blank' or '_self' (default: '_self').

In Chromium, prerendering requires knowing the target window or tab. target_hint lets Chrome identify the target before prerendering begins.

eagerness(Optional)

A string that hints to the browser how aggressively to prefetch or prerender link targets, balancing performance gains against resource use.

See more details: https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/script/type/speculationrules#eagerness

tag(Optional)

A string identifying a rule or ruleset. It is included in the Sec-Speculation-Tags request header for all speculations under that rule.

expects_no_vary_search(Optional)

This parameter should be the same as 'No-Vary-Search' parameter returned from the server.

If there is no 'No-Vary-Search' header present in the response, then the parameter makes no sense.

See more information about No-Vary-Search here: https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/No-Vary-Search

A string providing a hint to the browser as to what No-Vary-Search header value will be set on responses for documents that it is receiving prefetch/prerender requests for.

The browser can use this to determine ahead of time whether it is more useful to wait for an existing prefetch/prerender to finish, or start a new fetch request when the speculation rule is matched

isHTMLSupportSpeculationRule

A functions returns a boolean value indicating whether speculation rule is supported in browser.

import { isHTMLSupportSpeculationRule } from '@traveloka/core';

if (!isHTMLSupportSpeculationRule()) {
  // call fallback optimization method
}

specRuleHelper

import { specRuleHelper } from '@traveloka/core';

const {
  getActivationStart,
  isPrerendering,
  isSpecRulePage,
  pageActivated,
  pageActivatedWrapper,
} = specRuleHelper;

getActivationStart

A function returns the span between when a document starts prerendering and when it is activated.

If the value is 0, it means either:
- the page is not prerendered.
- the page is prerendering but not activated yet.

isSpecRulePage

A function returns boolean value indicates if the page is created by speculation rule prerender

isPrerendering

A function returns a boolean value indicates whether the current page is prerenderin in the background. Once user navigates to the page, this value will turn to false

pageActivated

A function returns a promise. The promise is resolved when a prerendered page is activated, or if it is not a prerendered page, or if it is already activated, then the promise is resolved immediately.

const { pageActivated } = useSpeculationRuleContext();

// example: defer external pageview tracking
useEffect(() => {
  const trackFn = async () => {
    await pageActivated();
    trackExternal('GTM', 'PAGE');
    trackExternal('FACEBOOK', 'PageView', undefined, {
      overrideProduct: 'default',
      overrideCountry: environment,
    });
    trackExternal('TIKTOK', 'pageView');
  };

  trackFn();
}, [trackExternal, environment]);

pageActivatedWrapper

A function wrapper to defer function invocation until the page is activated, or if it is not a prerendered page, or if it is already activated, then the function is fired immediately.

// example: defer flight search API call until page is activated
const fetchSearchResult = pageActivatedWrapper(
    useAPI<PublicApiRequest<Partial<SearchResultRevampResponse>>, PublicApiResponse<SearchResultRevampRequest>>({
      method: 'post',
      domain: 'flight',
      path: opts.isInitialFetch ? apiRtRevampInitial : apiRtRevampPoll,
    })
);

useEnableSpeculationRule

A feature control hook controls whether to turn on or off the speculation rule for a specific path.

If you are a product developer and you want to add a new speculation rule page. You should go to Web Feature Control page to add actions in Speculation Rule rule.

Initially, there are two actions under this rule:

- (Enabled) featureId: speculation-rule; enabled: true;
- (Text) featureId: speculation-rule; key: flightSearch; value: ENABLE;

usage

const fcSpeculationRule = useEnableSpeculationRule('flightSearch');

// example: turn on speculation rule for flight search prerender page when feature control is switched on
if (fcSpeculationRule.enabled) {
  return (
    <InjectSpeculationRule
      actionType="flightSearch"
      rules={{
        prerender: {
          url: searchLink,
          target_hint: '_self',
        },
      }}
    />
  );
} else {
  return null;
}

Notices

While Speculation Rule is easily to implement, since the timing of fetching and rendering the page is different from the normal case, it needs to be used with caution.

See more details on the risk and mitigation on this documentation: https://docs.google.com/document/d/1spkQ8nrT9Cjgg1NqXR-TXzHepStg_81iy6Jy-daftC4/edit?tab=t.0#heading=h.7awx244p14dh

Enablement of speculation rule​

Turn on feature control​

Check browser compatibility​

Inject Speculation Rule​

Defer actions​

Inspection​

Local Dev Environment​

Metrics​

Hit Rate​

Trackings​

Server Observability​

Performance measurement​

API​

InjectSpeculationRule​

actionType​

url​

target_hint(Optional)​

eagerness(Optional)​

tag(Optional)​

expects_no_vary_search(Optional)​

isHTMLSupportSpeculationRule​

specRuleHelper​

getActivationStart​

isSpecRulePage​

isPrerendering​

pageActivated​

pageActivatedWrapper​

useEnableSpeculationRule​

usage​

Notices​