.png){kind=icon}
Hi, my name is Joe Friesen and I'm a senior software developer here at PLUS QA. My principle responsibility is for development and maintenance of our Test Platform application, which is a Ruby-on-Rails based full-stack web application that utilizes React-on-Rails for rendering our views.
At PLUS QA, we own a large number of real, physical devices across a number of different platforms so that we can test applications in as many environments as possible. These devices are securely maintained at our Test Lab and checked out to testers throughout the day, depending on the needs of a project. One of the primary features of Test Platform is our Device Lab which helps us manage our test devices and seamlessly facilitate the checkout process.
However, due to the sheer size of our device library and the complexity of functionality in place, we began to notice a significant impact to page performance that would require addressing in order to improve the end user experience. There was also an opportunity to refactor code to make it easier to read and maintain in the future.
Performance and code maintainability
The first thing that jumps out: this page is slooooooooow. Load times for just the Device Lab page are often over 20 seconds just to first paint, which from a user experience perspective is not great. The majority of that time is spent in the device_lab_controller#index controller action, which (1) queries our database for (among other data needed for the front end operation) our entire device list, (2) serializes our data as JSON via ActiveModelSerializer, and (3) renders our React component DeviceLab as the container component for this view, passing the serialized data constructed in the controller to the React component as props.
This last bit, where we served data directly to the component via props, leads us to our second issue of code maintainability. Once we make our queries and provide the data to our component as props, we can't change or manipulate that data on the front end -- in React-land, props are immutable. This means copying the inherited props values to local state managed in our container DeviceLab component, and sharing that data with all the child components that need it. The result is the following:
// app/javascript/components/device_lab/device_lab.jsx (excerpted)
class DeviceLab extends React.Component {
constructor(props) {
super(props)
this.state = {
data: this.props.data,
currentUser: this.props.current_user,
userDevices: this.props.current_user.devices,
deviceState: this.props.current_user.role !== "client",
showStats: this.props.current_user.role === "client",
settings: this.props.settings,
modalOpen: false,
searchData: this.props.data,
isEditing: false,
searchTerm: "",
selectedPlatforms: [],
selectedFilters: [],
// and it goes on from there....
}
// ...
}
...
}
DeviceLab.jsx React component, with data management all happening in a mountain of local component state.
Our front end has data, user data, application settings, searches and filters, UI state, etc., all living in the local state of this one mega-component. And if we want to mutate the data by, for example, checking out a device, we have to make an API request in one of our child components, handle errors appropriately, then if the request resolves to status: :ok, we need to make sure our child component's response handler can lay its hands on the mega-component's this.setState class method and manually update the appropriate state in place.
The result is a tangled mess of props inheritance, unanticipated bugs, and a fundamental disconnect between server- and client-side data -- we no longer can say our React component relies on a single source of truth for its data.
State management: React-Redux in a React-on-Rails application?
So, at this point, if you have much familiarity with React front-end development, several issues should stand out right away. First and foremost, we have one big class component governing everything. So, that's the first step, let's turn this class component into a functional component by making this.state into separate state slices via useState, etc. This is great, but it doesn't really address the larger issue: the big component is too large to do its state management with local state, it needs to juggle lots of different state slices and their methods for updating them, and do a lot of props-drilling to make sure child components inherit data and the methods for updating it.
The solution for this is to implement an application-wide state management tool like Flux or Redux–personally, my experience is with Redux and Redux-based tools so that's what I prefer to use, and therefore is what we'll use for the rest of this refactor. Then, when we lay hands on our data, we stash it in the Redux store, build our actions for fetching and mutating the data, and provide the selectors for querying the store. Then, this allows our React components to be simplified. They no longer need to rely on long manifests of props inheritance or have to do data fetching, they simply query the store for the data they need and can focus on providing good UI elements.
But we have an issue: our application is a React-on-Rails application, and is not strictly speaking a Single-Page Application (SPA). We don't have one big component tree that we can wrap the whole of in a Redux provider -- we make a request to the /device_lab view, the controller serves up the DeviceLab component, and then if we navigate to a different page (say, /bugs) the DeviceLab component is unmounted and the bugs_controller#index serves a separate view. There's no common component tree and the Bugs React component will know nothing about the previous DeviceLab component.
What we can do, however, is build our Redux store in much the same way as we would for an SPA, where we delineate store slices such as device_lab, bugs, et al. separately, but wrap each view-level component such as our DeviceLab component in a ReduxProvider component. Assuming we have already set up our Redux store in app/javascript/rdx/store.js, we write our provider and wrapper like so:
// app/javascript/rdx/utils/Provider.jsx
import React from "react";
import PropTypes from "prop-types";
import { Provider } from "react-redux";
import store from "../store";
const ReduxProvider = ({ children }) => {
return <Provider store={store}>{children}</Provider>;
};
ReduxProvider.propTypes = {
children: PropTypes.oneOfType([
PropTypes.node,
PropTypes.arrayOf(PropTypes.node),
]),
};
export default ReduxProvider;
// app/javascript/components/device_lab/index.jsx
import React from "react";
import ReduxProvider from "@rdx/utils/Provider";
import DeviceLab from "./device_lab";
const DeviceLabIndex = (props) => {
return (
<ReduxProvider>
<DeviceLab {...props} />
</ReduxProvider>
);
};
export default DeviceLabIndex;
A simple wrapper component for the Redux store provider, which will allow the underlying component to receive the props it expects from the Rails view while also allowing it to access the Redux store.
Now, any data we need to pass directly from the controller action to the component is done via the DeviceLabIndex's props, and the DeviceLab functional component has access to our Redux store.
There are two drawbacks with this setup however.
- The store is torn down and rebuilt going from one Redux-enabled view to another, e.g. if I navigate from
DeviceLabtoBugs, the store is unmounted and remounted from scratch. If this issue of one hand not knowing what the other is up to is a problem, we can have the interrelated components dispatch fetch actions to get fresh data, or, if you're using Redux-Toolkit (and we will be), we can take advantage of persistence and rehydration: https://redux-toolkit.js.org/RTK Query/usage/persistence-and-rehydration. For our purposes, we won't need this feature. - Application-wide settings and data such as user sessions etc. are not shared as a result. This again for our purposes is a trade-off worth making, it is preferable to leave application-wide data as a concern of the larger
ApplicationController, and here by bifurcating the flow of props with theDeviceLabIndexcomponent we can let application-defined data to flow to the component as before without disruption.
Redux-Toolkit, RTK Query
Great, we now have a view-level component hooked up to our Redux store, and any child components that need device data can get it via selectors without having to rely on inheriting the right props. Now we need to hook up our front end store with the backend API and build out our deviceLab store slice. To do that, we will rely on Redux-Toolkit for the store slice and RTK Query for handling our data queries and mutations, which will allow us to get the ad-hoc API requests out of the components and centralized in our store slice.
First, we follow the steps for installing Redux-Toolkit and RTK Query:
yarn add @reduxjs/toolkit
Then, we define our API slice, this will let us define the testPlatformApi and help us define what all requests should look like. Then we construct our rootReducer and give it to the Redux store:
// app/javascript/rdx/slices/api.js
import { createApi, fetchBaseQuery } from "@reduxjs/toolkit/query/react";
const testPlatformApi = createApi({
reducerPath: "testPlatformApi",
baseQuery: fetchBaseQuery({
baseUrl: "/",
prepareHeaders: (headers) => {
const csrfToken = document
.querySelector("meta[name='csrf-token']")
.getAttribute("content");
headers.set("Accept", "application/json");
headers.set("X-CSRF-TOKEN", csrfToken);
return headers;
},
}),
endpoints: (builder) => {
return {
// (1) list all API endpoints here
};
},
// (2) add RTK Query tags
tagTypes: [],
});
export default testPlatformApi;
export const {
// (3) export all query and mutation actions here
} = testPlatformApi;
// app/javascript/rdx/slices/rootReducer.js
import { combineReducers } from "@reduxjs/toolkit";
import testPlatformApi from "./api";
const rootReducer = combineReducers({
[testPlatformApi.reducerPath]: testPlatformApi.reducer,
// (4) all other reducers go here
});
export default rootReducer;
// app/javascript/rdx/store.js
import { configureStore } from "@reduxjs/toolkit";
import { setupListeners } from "@reduxjs/toolkit/query/react";
import rootReducer from "./slices/rootReducer";
import testPlatformApi from "./slices/api";
const store = configureStore({
reducer: rootReducer,
middleware: (gDM) => {
return gDM().concat(testPlatformApi.middleware);
},
});
setupListeners(store.dispatch);
export default store;
The beginning of our new RTK Query API slice, that will connect front-end Redux store data with response data from our Test Platform Rails API backend. We will fill in the blanks (1)-(4) next.
In testPlatformApi, we construct the API slice, which will include the headers common to all requests that will be made with this API. We state our baseUrl as the path /, our API is namespaced to the root path, but if your backend API endpoints are namespaced to a sub-path such as /api or is on a different URL altogether from your front-end, you would change that here. Then, we have to state our headers as only accepting JSON responses and also provide our CSRF token as well -- all of these can be added to or overridden per endpoint but this will work for us most of the time. In doing this ad hoc in the components, using a library like axios or the built-in fetch API we'd have to do this every time we want to hit a backend endpoint, and that gets old fast.
Next, we need to add a device_lab slice. To do this, we need to (1) add the device_lab endpoints and (2) tags to testPlatformApi, (3) destructure the query and mutation actions generated by createApi from testPlatformApi and make them available for export, and (4) add the deviceLab slice's reducer to the rootReducer. That will look something like this:
// app/javascript/rdx/slices/api.js
import { createApi, fetchBaseQuery } from "@reduxjs/toolkit/query/react";
import { deviceLabEndpoints, DEVICE_LAB_TAGS } from "./deviceLab";
const testPlatformApi = createApi({
reducerPath: "testPlatformApi",
baseQuery: fetchBaseQuery({
baseUrl: "/",
prepareHeaders: (headers) => {
const csrfToken = document
.querySelector("meta[name='csrf-token']")
.getAttribute("content");
headers.set("Accept", "application/json");
headers.set("X-CSRF-TOKEN", csrfToken);
return headers;
},
}),
endpoints: (builder) => {
return {
// (1) list all API endpoints here
...deviceLabEndpoints(builder),
};
},
// (2) add RTK Query tags
tagTypes: [...DEVICE_LAB_TAGS],
});
export default testPlatformApi;
export const {
// (3) export all query and mutation actions here
// these are generated by createApi from the actions we will define
// next in our deviceLab slice
useGetDeviceLabIndexQuery,
useGetUserDevicesQuery,
useCheckOutDeviceMutation,
useCheckInDeviceMutation,
} = testPlatformApi;
// app/javascript/rdx/slices/rootReducer.js
import { combineReducers } from "@reduxjs/toolkit";
import testPlatformApi from "./api";
import deviceLabReducer from "./deviceLab"
const rootReducer = combineReducers({
[testPlatformApi.reducerPath]: testPlatformApi.reducer,
// (4) all other reducers go here
deviceLab: deviceLabReducer,
});
Filling in the blanks (1)-(4), adding in the details of our yet to be written deviceLab slice to the testPlatformApi slice.
Finally, we're importing all this cool stuff from the deviceLab slice but we haven't defined what our deviceLab slice actually is yet. This is where we set up our one-to-one correspondence between our backend REST API routes, the redux actions that will call those routes, what to do with the response that comes back, handling loading and error states, and more.
// app/javascript/rdx/slices/deviceLab/index.js (excerpted)
import { createSlice } from "@reduxjs/toolkit
const INITIAL_STATE = {};
export const deviceLabSlice = createSlice({
name: "deviceLab",
initialState: INITIAL_STATE,
reducers: {}
});
// Tags
export const DEVICE_LAB_INDEX_TAG = "deviceLabIndex";
export const DEVICE_LAB_USER_DEVICES_TAG = "deviceLabUserDevices";
export const DEVICE_LAB_TAGS = [
DEVICE_LAB_INDEX_TAG,
DEVICE_LAB_USER_DEVICES_TAG,
];
// RTK Query endpoints
export const deviceLabEndpoints = ({ query, mutation }) => {
return {
// queries
getDeviceLabIndex: query({ // (1)
query: (q) => {
return {
url: `device_lab?${q}`, // (2)
}
},
providesTags: [DEVICE_LAB_INDEX_TAG],
}),
getUserDevices: query({
query: (id) => `users/${id}/devices`,
providesTags: [DEVICE_LAB_USER_DEVICES_TAG], // (3)
}),
// mutations
checkOutDevice: mutation({
query: (body) => {
return {
url: `devices/${body.device_id}/check_out`,
method: "POST",
body,
};
},
invalidatesTags: [DEVICE_LAB_INDEX_TAG, DEVICE_LAB_USER_DEVICES_TAG],
}),
checkInDevice: mutation({
query: (body) => {
return {
url: `devices/${body.device_id}/check_in`,
method: "POST",
body,
};
},
}),
};
}
export default deviceLabSlice.reducer;
The deviceLab slice, where we fill in the details of how our RTK Query queries and mutations should interact with our backend. Here we can define request headers and body, manipulate the response data before it is saved to the store, and set up our cache invalidation scheme.
There's a lot going on here, and this is just scratching the surface of what we can do with RTK Query, but a few notes:
- Earlier, when we exported
useGetDeviceLabIndexQueryfrom thetestPlatformApislice, this was generated from thegetDeviceLabIndexquery we are defining here. The result will be a React hook that can be called in components or custom-written hooks, and will let you lay your hands directly on the data as well as the query and loading/error states of the action. Any query or mutation defined here in these endpoints will have such a hook automatically generated by RTK Query'screateApimethod in this fashion with theuse{...}Queryoruse{...}Mutationnaming convention. To see the generated hooks in action, check out the RTK Query documentation example here. - Each endpoint has a
urlvalue that it will make its requests against, built off of thebaseUrlspecified in the API slice, and urls are generated dynamically via this query method. So for the device index, the URL would bebaseUrl + endpointUrl, which for thegetDeviceLabIndexaction would be/device_lab?${q}. We append the?${q}to our url because we may want to include query parameters such as filter values and search, and this will append a query string to the end of our URL. - One of the cool features of using RTK Query is its automated caching and re-fetching logic. If my front end React component needs to make a request to
/device_lab?foo=barand RTK Query knows it has already made a request to that endpoint with that exact query string, instead of making a redundant API call to get the same info again, it will preempt the request and serve cached data corresponding to that URL instead.
But, suppose we make a mutation—for example, if we decided to check out a new device and then made the same request, the cached data would be stale; the index list will show that device as being available even though it's been checked out since that request was made. This is what the providesTags and invalidatesTags help us with. By linking the getDeviceLabIndex query with the checkOutDevice mutation via the DEVICE_LAB_INDEX_TAG, any time we call the checkOutDevice mutation, RTK Query will invalidate the device lab index cache and force a refetch for fresh data. Again, we're just scratching the surface of what we can do with the caching features here, we can get much more granular with tags and invalidating behavior. For more information on the ways the cache invalidation scheme can be restructured, see the RTK Query documentation here.
Conclusion
We’ve now set ourselves up for success to refactor our overgrown React DeviceLab component, we can begin to break down its use local state and let all of the DeviceLab’s child components remove the tangle of props inheritance and locally-defined API request handlers, and let them subscribe to the Redux store slice for bakery-fresh data. When this is done, tracking down strange UI bugs, adding new features, and writing tests all become significantly easier and quicker.
However, we haven’t yet addressed the main issue: poor UX due to long page load times. In the next part, we’ll turn our attention to the Rails backend, where we will refactor our controllers for DRYer more efficient database queries, cache long-term unchanging data in Redis, paginate our table, and add scopes for easy filtering and searching. Until next time!