Building Job URL Extraction into JobTrackr

I wanted the feature to fit naturally into the existing application form rather than introduce another workflow.

The experience I had in mind was simple.

Paste a job URL, let JobTrackr retrieve the relevant details, review or correct whatever comes back, and save when you’re ready.

If extraction doesn’t work, you can still enter the information yourself.

One thing I was particular about was keeping the user in control.

An extracted job title might not be the title someone wants to use in their tracker. A company might appear under a different name on the job board. Some fields might be missing altogether.

So I didn’t want extraction to automatically create an application or make decisions on someone’s behalf.

It needed to assist, not take over.

That decision shaped quite a bit of the implementation.

This was a deliberate decision.

I didn’t want to use AI for job details extraction.

Not because I’m against AI or wouldn’t consider introducing it into JobTrackr. I’ve just been intentional about what I add to the application.

I don’t want to introduce an AI-powered workflow simply because I can. There has to be a genuine need for it, and it has to fit naturally into the product.

For this feature, I felt there was a more straightforward approach worth exploring first.

A lot of job posting platforms already expose structured information. Some have public endpoints for retrieving job details, while others include structured data in their pages to help search engines understand the content.

That gave me somewhere to start.

I implemented provider-specific extraction for Greenhouse and Lever, alongside support for JobPosting structured data, commonly represented as JSON-LD.

JSON-LD is essentially a way for websites to describe their content in a format that software can understand. For a job posting, that might include the title, hiring organization, location, employment type, or salary.

Where the more structured approaches don’t provide enough information, JobTrackr can also inspect limited page metadata, such as a title or description. Those less certain values are presented as suggestions rather than automatically filling the form.

The idea was to work with information that was already available instead of introducing a model to interpret every posting.

I also had to be realistic about the scope.

JobTrackr is still a relatively small product, with fewer than 100 users at the moment. I didn’t see the need to build an extraction system that tried to support every job board from day one.

Start small, see how it performs, and expand when there’s a reason to.

Of course, that meant accepting that not every URL would work.

Some websites don’t expose useful structured data. Others require authentication, block automated requests, or rely on JavaScript to display important information.

And even when a page contains structured data, it isn’t always complete or reliable.

Which raised another question: what should happen when JobTrackr can retrieve some information, but not everything?

On the surface, extraction sounds pretty straightforward.

Fetch a page. Find the relevant information. Put it into the form.

But the more interesting engineering challenges came from thinking about what someone might be doing while that process was happening.

Say you paste a URL and immediately start typing the company name yourself.

What happens if extraction finishes a second later and returns a different company name?

Should it replace what you typed?

I didn’t think so.

The same goes for deliberately clearing a field. An empty field doesn’t always mean someone forgot to fill it. Sometimes they simply don’t want that value.

So I needed to distinguish between information JobTrackr had filled automatically and information the user had deliberately entered or changed.

The implementation keeps track of those differences.

When extraction returns, it can populate eligible blank fields without overwriting fields the user has manually touched. Less certain information is presented as suggestions that can be accepted or dismissed.

It might seem like a small interaction detail, but it makes a difference to how the feature feels.

There was another scenario to consider.

What if someone pastes one URL, changes their mind, and pastes another before the first extraction finishes?

The second request might finish before the first.

Without handling that properly, an older response could end up filling the form with details from the wrong job.

I handled this by tracking which extraction request belongs to the current form state and ignoring responses that are no longer relevant.

The user doesn’t need to know any of that is happening.

They should just be able to use the form without unexpected values appearing or disappearing.

There was another side to this feature that wasn’t immediately visible in the interface.

JobTrackr was now accepting URLs from users and retrieving information from external websites.

That introduces risks beyond whether extraction succeeds.

For example, the server shouldn’t be allowed to follow an arbitrary URL into a private network or internal service. It also shouldn’t follow redirects endlessly or try to process an unexpectedly large response.

So I needed safeguards around which destinations could be accessed, how redirects were handled, how much information could be downloaded, and how long retrieval could take.

I also added rate limiting and shared request controls to keep extraction bounded.

The initial allowance was 10 attempts per rolling hour. I later increased it to 20 after the earlier limit proved restrictive for people processing multiple applications.

Most of this isn’t visible when someone pastes a URL into a form.

But it’s part of what makes a feature ready for real usage, rather than something that only works under ideal conditions.

The first production rollout didn’t go as planned.

After merging the feature and deploying it, one of the smoke tests timed out during verification. I rolled back the deployment rather than leave the new release running without completing the checks.

At that point, I still needed to understand what had gone wrong.

What I eventually found was interesting.

The test was supposed to replace one job URL with another. Instead, its simulated paste action inserted the second URL into the existing value, producing a combined URL that wasn’t a valid job posting path.

The extraction feature handled it as unavailable, but the test continued waiting for a suggestion that wasn’t supposed to appear.

Eventually, it timed out.

The problem was in the test interaction and expectation, not the extraction logic I’d initially been investigating.

The fix was to make the URL replacement explicit and verify the behaviour that actually mattered: the request settles, the form remains usable, and manual entry still works.

That wasn’t the only issue during release verification. A separate Lever request also experienced a genuine timeout, which reinforced the importance of distinguishing unreliable external responses from problems within the application.

The feature was subsequently released successfully.

The experience reinforced something important for me: reliable tests and useful diagnostics matter just as much as getting the implementation right.

A failing test tells you something needs attention. It doesn’t automatically tell you what’s broken.

And when external websites are involved, there’s a difference between testing whether your application behaves correctly and expecting a third-party page to return exactly the same information every time.

That’s a distinction I appreciate more now.

After getting extraction into production, I wanted to understand what was actually happening when people used it.

I already had an Analytics/Insights module in JobTrackr tracking different aspects of the application, so extending it to cover extraction made sense.

It wasn’t part of the initial implementation. I added it afterwards because I wanted a better way to evaluate the feature beyond my own testing.

Testing it myself could tell me whether it worked for the URLs and scenarios I tried.

But that alone couldn’t tell me whether people were actually using it, where they were running into problems, or whether extraction attempts were leading to saved applications.

That was what I was most interested in.

The additional tracking records extraction outcomes, how long attempts take, which fields are returned, and whether an attempt is later associated with a saved application.

It also distinguishes between information that was extracted and information the user had to complete manually.

In one early 30-day snapshot, JobTrackr recorded 28 extraction attempts. Of those, 27 returned usable information, including partial results, and 19 were associated with saved applications.

Those are encouraging observations, but it’s still a small sample. Attempts aren’t unique users, and a saved application doesn’t necessarily mean every extracted value was used.

The numbers also don’t tell me whether extraction is faster than manual entry. I don’t have that comparison yet.

And that’s fine.

I wasn’t expecting a handful of early numbers to determine whether the feature was successful.

I wanted visibility into what was happening beyond my own testing.

Now I have a starting point.

There’s still room to improve job URL extraction. I’d like to support more posting platforms and make partial results more useful.

AI-assisted extraction is something I might explore eventually, but I don’t see a reason to introduce it just yet.

For now, I want to see how people actually use the feature before deciding what to build next.

Job URL extraction is available in JobTrackr. Paste a job posting link, review the details it can retrieve, and add the application to your tracker.

Try JobTrackr (opens in a new tab)