00:00:02 In this lesson, let's implement workflows.

00:00:05 A workflow is basically like a message queuing or task scheduling system, where the client or an API request triggers a specific workflow to happen.

00:00:15 In our case, we want to schedule a lot of email reminders before our users hit their subscription deadlines, giving them more time to think about whether

00:00:24 they want to cancel their subscription or keep it going.

00:00:27 And for that, we'll use UpStash.

00:00:29 Before we start implementing it, I wanted to quickly go over our workflow algorithm, or in other words, how will our workflows be triggered and how will

00:00:38 they behave.

00:00:39 Step one is, of course, to trigger the workflow.

00:00:42 The workflow begins whenever a user creates or submits a new subscription.

00:00:46 we're going to pass the created subscription ID to our workflow.

00:00:50 After that, we want to retrieve the subscription details.

00:00:53 Specifically, we want to extract the subscription ID and then search for the corresponding subscription in the database.

00:01:00 Finally, we'll run some checks to see if a subscription doesn't exist, then we have an error and we kill the process.

00:01:06 But if a subscription does exist, we'll check its status.

00:01:10 If it's inactive, then we're going to kill the process.

00:01:13 But if it's active, then we want to check out the renewal date.

00:01:17 If the renewal date has passed, then we're going to log this information and exit.

00:01:22 But if the renewal date is in the future, the reminder loop begins.

00:01:26 And then we can schedule predefined reminders, for example, 3, 7, or 14 days before subscription deadline, and send those emails.

00:01:34 And this process will repeat for all the reminders in the list.

00:01:37 And after it's done, the workflow has done its job.

00:01:40 Let me show you just how simple it is to implement it.

00:01:42 We'll use upstash.

00:01:44 So if you haven't already created an account, you'll have to create one now.

00:01:47 Click the link in the description to be able to follow along and see exactly what I'm seeing.

00:01:52 Head over to login or create a new account and choose one of the options.

00:01:56 Head over to workflow and copy those environment variables.

00:01:59 I'll copy the qstash URL.

00:02:01 and head to my.env.development.local where I'll say upstash.

00:02:06 And I will render an upstash underscore URL and make it equal to the string that I just copied.

00:02:11 We also want to get the QStash token.

00:02:14 Oh, that's my bad.

00:02:14 This was supposed to say QStash, not upstash.

00:02:18 This will be QStash token.

00:02:20 And finally, we have the signing keys.

00:02:22 So let's copy both of these.

00:02:24 I'll copy first the name, paste it here, and I'll copy the second name.

00:02:28 And now we can add those two keys.

00:02:31 There we go.

00:02:32 It should look something like this.

00:02:33 And now that we have our ENVs, we can install upstash client.

00:02:37 I'm going to do it by following the quick start for Express.

00:02:41 And the first step is to install upstash workflow.

00:02:44 So I'll copy this command without the bash at the start.

00:02:47 Just npm install at upstash forward slash workflow.

00:02:51 We have to set up our envs, which we have already done, and we won't necessarily follow this entire tutorial.

00:02:56 I mean, that will kind of defeat the purpose of you watching this video.

00:03:00 So let me teach you exactly what we have to do next.

00:03:04 We'll keep our code super clean by creating an additional configuration file.

00:03:08 You can see how much sense it makes to separate all the configs into their own files.

00:03:13 I'll call it upstash.js.

00:03:16 Here, I will import client as workflow client coming from at forward slash up stash forward slash workflow.

00:03:28 And I'll also import our ENVs.

00:03:30 Oh, I think I forgot to add them in our dot ENV config.

00:03:33 So right here, I'll get the Q stash underscore token, as well as the Q stash underscore URL.

00:03:40 We'll only need those two for now.

00:03:42 So here we can import them by importing Q stash underscore token, as well as Q stash underscore URL from dot slash ENV dot JS.

00:03:53 And finally, let's simply export const workflow client is equal to new workflow client.

00:04:01 And to it, we have to pass an object within which we can specify a base URL of Q stash URL, as well as the token equal to Q stash token.

00:04:11 That's it.

00:04:12 That's the config.

00:04:13 And now we can create a new file for running all kinds of different workflows.

00:04:18 Let's do it under routes and create a new route called workflow.routes.js.

00:04:25 Within here, we can import router coming from Express.

00:04:31 We can initialize a new workflow router equal to router, which we call like this.

00:04:39 And we can define a route, for example, a basic get route for now.

00:04:44 And finally, don't forget to run export default workflow router.

00:04:50 That way we can use it within our app.

00:04:52 Of course, we have to replace this with real workflow logic.

00:04:56 So let's head over into the app.js and right below the subscriptions, let's duplicate it and call it workflows.

00:05:05 And just call the workflow router and make sure that it's coming from workflow routes.js.

00:05:10 Now, as you know, every routes file has to have its own controllers file.

00:05:15 So create a new file called workflow dot controller.

00:05:21 Within here, we can create a function which will be responsible for sending reminders by saying export const send reminders.

00:05:30 And this one will be different from a typical rec and res route.

00:05:35 This one will be wrapped within a serve.

00:05:38 This serve is coming from upstash workflow express.

00:05:42 So we can say const destructure the serve and say require at upstash forward slash workflow, forward slash express.

00:05:54 And when packages are written as ESM modules, you could automatically use import to import them.

00:06:01 But because upstache workflow was likely written using common.js, import won't work directly, so we need to use require to import it.

00:06:10 But in our package.json, we have switched type to module, which means that we only can have imports.

00:06:17 So one way to go over that and still allow this single require statement is to say import destructure create require and that's coming from module and

00:06:31 then we can say const require is equal to create require And import that meta.url.

00:06:38 And this will allow us to nicely import this upstache workflow.

00:06:42 Now we are wrapping this function with a serve.

00:06:45 And the only thing it'll have is an async function that'll give us access to the context of that workflow.

00:06:51 And we can then open up a new function block.

00:06:55 That'll look something like this.

00:06:57 Within here, we first want to extract the subscription ID from a specific workflow.

00:07:04 So let's say subscription ID is equal to context.requestPayload.

00:07:11 So when we trigger a specific workflow, we will pass the ID of the subscription that workflow is for.

00:07:19 Then, let's fetch the details about the subscription by saying const subscription is equal to await fetch subscription.

00:07:29 And to it, we can pass the context as well as the subscription ID.

00:07:35 And this fetchSubscription function is a new function we can create below by saying const fetchSubscription is equal to async.

00:07:45 We'll accept the context and the subscription ID.

00:07:49 And we'll return await context.run.

00:07:53 So here we're starting the context for getSubscription.

00:07:58 And when it happens, we'll open up a new function block.

00:08:02 where we'll simply return subscription.findById, and we'll pass over a subscription ID, and we will populate it with user information,

00:08:13 specifically name and email belonging to that user.

00:08:17 It looks something like this.

00:08:19 Make sure to have these two return keywords at the start of this code block and at the start of this one, because otherwise functions return undefined

00:08:27 if you don't explicitly return something.

00:08:29 Great.

00:08:30 So now we're starting the context of getting the subscription and we're returning it right here.

00:08:35 Next, we'll check if there is no subscription.

00:08:39 or if the subscription.status is not equal to active, then our job is done here.

00:08:46 We can kill the workflow.

00:08:47 So let's simply say return, meaning exit out of this function.

00:08:51 Don't send a reminder.

00:08:54 Else, we have to figure out when is the renewal date.

00:08:57 So let's say const renewal date is equal to And now we could use the regular JavaScript date object and then wrap a renewal date with it,

00:09:08 but we'll have to do a bit more date and time calculations here.

00:09:12 And for that, I prefer to use a very lightweight package called dayjs.

00:09:17 So in my terminal, I'll say npm install dayjs and let's import it at the top.

00:09:23 I'll do it right here by saying import dayjs from dayjs.

00:09:30 Oh, and don't forget to import the subscription because we're using it right here.

00:09:34 So that's going to be import subscription coming from models, subscription, model.js.

00:09:40 And we can have it below because it is a local import.

00:09:44 Great.

00:09:45 So now we can wrap our renewal date with day.js object.

00:09:50 subscription.renewaldate, and then we can have another check and see if renewal date is before, and then we can just pass dayjs and call it.

00:09:59 If you call it like that, it basically returns the current date and time.

00:10:02 So we're checking if renewal date is before the current date and time.

00:10:06 If it is, well, then we're going to simply return a console log saying something like renewal Date has passed for subscription with this ID.

00:10:17 And we can also add stopping workflow at the end.

00:10:22 Nothing to do there.

00:10:23 And we can just return, meaning exit out of this workflow.

00:10:27 Finally, now that we have figured a criteria for when we want to exit out of the workflow, let's actually figure out what the workflow will do once we're

00:10:35 in it.

00:10:36 We can do that by creating a list of different reminders for days when we want to send them.

00:10:41 So I'll say const reminders is equal to an array.

00:10:46 And we can do something like seven days ahead, five days, two days, and maybe one day.

00:10:52 So we have different reminders.

00:10:54 After that, we can open up a new for loop and say for const days before of reminders.

00:11:06 So from each reminder, we'll get the number of days before when we want to send that reminder.

00:11:11 And then we'll say const reminderDate is equal to renewalDate.subtract.

00:11:19 And we're going to subtract the number of days before just like this by saying reminder date is renewal date and then subtract days before.

00:11:30 So let me give you an example of that.

00:11:32 Let's say that the renewal date is maybe 22nd of February.

00:11:38 In that case, the reminder date is 15th of Feb, which is seven days before.

00:11:42 But then we also have a second reminder, which is five days before.

00:11:46 So that's going to be the 17th.

00:11:48 Then we're going to have a 20th and finally one day before on the 21st.

00:11:52 So we're going to be in a loop each time for each one of these reminders.

00:11:57 Then we want to figure out if reminder date that is after The current date and time, so day.js, that means that we want to put it to sleep until it's ready

00:12:09 to be fired.

00:12:10 We can do it by creating a new sleep function.

00:12:13 I'll create it right below and I'll call it const sleep until reminder.

00:12:19 And it'll be equal to an asynchronous function that accepts the context, the label it wants to fire, as well as the date.

00:12:28 And here we can simply console.log and say something like sleeping until label reminder at specific date.

00:12:37 And more importantly, we can await context.sleepUntil this specific label and this specific date.

00:12:47 So date.toDate.

00:12:50 Hopefully that makes sense.

00:12:51 We're putting that reminder to sleep until it's time to trigger it, for which we can have another function const trigger reminder.

00:12:59 We'll make it equal to a function that accepts the context as well as the label.

00:13:05 And then we're going to return await context.run for this specific label.

00:13:11 And it'll run the following code.

00:13:13 For now, it'll simply say console.log triggering label reminder.

00:13:19 And then later on, we can send an email here.

00:13:23 Or it doesn't even have to be an email.

00:13:25 It can be an SMS.

00:13:26 It can be a push notification.

00:13:29 It can really be anything you want to do.

00:13:31 You can run any kind of custom logic whenever this reminder is triggered.

00:13:36 But keep in mind, this don't have to be just reminders.

00:13:39 This could be anything you want to run periodically or you want to run based on some kind of algorithm or logic.

00:13:45 You are in control.

00:13:48 We'll figure out how to send this email very soon but for now let's put this reminder to sleep when it needs to sleep and let's trigger it when it needs

00:13:55 to be triggered.

00:13:57 In this if statement I'll say await sleep until reminder and I'll pass a context to it and I'll pass the label.

00:14:07 The label will look something like this reminder days before, days before.

00:14:14 This is going to be the actual number like five days before.

00:14:18 And then finally I'll pass the reminder date.

00:14:21 Outside of this if statement, we're actually ready to trigger the reminder.

00:14:25 So we'll say await trigger reminder.

00:14:27 We're going to pass in the context and we'll say reminder for five days before, for example.

00:14:33 So let's figure out these functions.

00:14:35 First, we're fetching the subscriptions.

00:14:37 We're checking if that renewal date is maybe in the past or if the subscription doesn't exist or if the status is not active.

00:14:43 In all of these cases, we just stop the reminders from going.

00:14:47 But once we figure out that it is an active subscription and it has a renewal date that is in the future, we map over all of the reminders that we specified,

00:14:55 like five days before, seven days, two days, one day, And then we figure out when to put it to sleep and when to actually trigger it.

00:15:03 And we use these helper functions such as sleepUntil or trigger to make it do its job.

00:15:09 And let's trigger the workflow.

00:15:11 We'll do it right after the subscription gets created.

00:15:14 The only thing we have to do is say await workflow client.

00:15:20 This is coming from upstash.

00:15:22 So import it at the top dot trigger.

00:15:25 And we're going to pass in an object of additional info for this workflow we want to trigger.

00:15:32 First, it needs a URL.

00:15:33 So which API endpoint do we have to call to be able to run this workflow?

00:15:38 It'll be starting with server.

00:15:42 underscore URL, and you'll have to add it over in your env.development.local.

00:15:48 So scroll to the top and if you haven't already, and I don't believe you have, you can add this server underscore URL is equal to,

00:15:57 and put your current localhost address, HTTP colon forward slash forward slash localhost 5500. Usually you would have to deploy the entire app and use

00:16:08 a production URL to test any kind of scheduling type of workflow.

00:16:12 or use something called ngrok to do it.

00:16:14 It has a super complex setup.

00:16:17 But luckily for us, AppStash provides an easy local development server that won't just save you time and a lot of headache from going through complex setups,

00:16:26 but also save you money from running all of those test workflows.

00:16:30 You have unlimited tests here.

00:16:31 So, let me show you how to do it.

00:16:33 I'll open up a third terminal, which will run our local development server.

00:16:38 And for now, I'll run npx at upstash forward slash qstash dash cli dev.

00:16:45 It'll ask us whether we want to install that package.

00:16:47 I'll say, why?

00:16:48 Go ahead.

00:16:49 As soon as you do that, in the terminal, you'll get a URL and a dummy token.

00:16:53 Copy those and add them over to .env.development.local.

00:16:59 Let's add them below current UpStash ENVs, and then move the original ones over to our second .env file, .env.production.local.

00:17:10 So this is gonna be UpStash, and I'll paste them right here.

00:17:13 Oh, and this is important.

00:17:14 Check out this sample curl request that UpStash has given us to test this out.

00:17:19 Hidden within it, there's a QStash URL.

00:17:22 So copy this starting part of this up to the 8080 part, and then head back to the development part and say Q stash underscore URL and make it equal to

00:17:33 this HTTP colon forward slash forward slash 127001880. So now we have two different instances of UpStash.

00:17:43 One for testing in development and one for production.

00:17:46 Also, in production, we need these Q stash current and next signing keys, but in development, we don't actually need them.

00:17:54 so I will completely remove them from development.

00:17:57 You need to remove them in order for everything to work.

00:18:00 Let's actually copy this entire curl request, and let's leave this third terminal to be running.

00:18:05 I'll rename it as upstash.

00:18:07 The first one will be our own server, so I will rename it to server.

00:18:12 And within this terminal, which I'll simply call terminal, we can run some additional commands, such as running this curl command that was given to us

00:18:21 by UpStash.

00:18:23 I'll press Enter, and you can see that we get back a message ID of message 2kxd.

00:18:29 This is a random workflow ID generated for us in the terminal, which means that our workflows are indeed generating locally.

00:18:37 But let's not forget the workflow routes.

00:18:40 If you think about it, we never actually got the opportunity to use the Send Reminders workflow that we created not that long ago.

00:18:47 Of course, this one right here is the one I'm referring to, Send Reminders.

00:18:51 So we'll call it once we hit this endpoint, which is supposed to trigger the workflow.

00:18:57 I simply have to say Send Reminders and import it from controllers, workflow controller.js.

00:19:02 But then when are we actually going to fire it?

00:19:05 Well, the answer lies in the subscription controller because here we have defined the URL.

00:19:11 So it's going to be api v1 workflows and then forward slash subscription forward slash reminder.

00:19:18 And it is a post request.

00:19:19 So let's say workflow router dot post forward slash subscription forward slash reminder.

00:19:25 And then we call this send reminders function.

00:19:28 Now you're ready to test it out.

00:19:30 Back in our HTTP client, I prepared my route of localhost 5500 API v1 subscriptions.

00:19:38 I have my bare token right here, as well as the body of the new subscription I'm trying to make.

00:19:43 This is a subscription for JS Mastery Pro Elite membership.

00:19:47 We have the price, the currency, and the category, as well as the start date.

00:19:52 So what do you say that we give it a shot?

00:19:55 I'll click send.

00:19:57 And we get a 201 created.

00:20:00 That's amazing.

00:20:01 But hey, this worked before.

00:20:03 This is nothing new.

00:20:04 So what matters more here is that now alongside the newly created subscription, we also get a workflow run ID.

00:20:13 It ends with O-N-G.

00:20:15 Okay.

00:20:16 So let's see if we get anything in the terminal.

00:20:18 Back into the code, if you go to the upstash terminal, you'll see that we have some kind of a message ID with a workflow run O-N-G.

00:20:27 But now if you head into your server terminal, it looks like we got a small type error.

00:20:32 This converting circular structure to JSON, especially when mentioning a Mongo client object, typically refers to forgetting to pass an async or an await

00:20:41 before some kind of a Mongoose action.

00:20:43 So if we head over into workflow controller and pay attention to the only Mongoose caller making here, subscription.findbyid,

00:20:52 we actually have to turn this function into an asynchronous function.

00:20:56 That'll resolve the error.

00:20:58 Also, if I scroll a bit above, you'll notice that here I said subscription status is not active, but this active is actually an undefined variable.

00:21:07 What I meant to do was say active within string size, because the status is literally a string of active.

00:21:14 So now that I fixed those, let's just render another subscription.

00:21:17 We get true, this time ending with 6KB.

00:21:20 And if I get back, you can see sleeping until, reminder five days before, and reminder will happen at, and of course this date and time will be different

00:21:29 for you.

00:21:30 So this is great.

00:21:31 It means our workflows are working, but just seeing these workflow IDs and console logs within Terminal is not that fun.

00:21:38 So let's head into the workflow controller.

00:21:42 And scroll down to the part where we left a comment.

00:21:45 When we trigger a reminder, we want to do something.

00:21:48 We want to send an email or an SMS or a push notification.

00:21:51 And that's exactly what we'll do in the next lesson.

00:21:54 Then we'll be able to utilize the full power of the flexibility of these workflows to trigger custom email reminders.

00:22:01 So, let's do that next.