https://rudyfaile.com/2020/07/06/building-an-easy-web-application/

Skip to content

  * Home
  * About
  * Contact
  * IRL Projects
      + 100 Days of Running
      + Motorcycle Exhaust
  * Hardware Projects
      + Automatic Fish Feeder
      + Crypto Miner
      + MacBook Pro
  * Software Projects
      + Draw!
      + Lucy's Adventure
      + 5/3/1 Calculator

  * Search

Search for: [                    ] Search
 

Rudy Faile

My life, interests, and work.

Portfolio, Software, Technology, Work

Building an "easy" web application

[cors-sucks]
Date: July 6, 2020Author: Rudy 2 Comments

I decided to spend my Fourth of July weekend transforming a small
Python utility I wrote a couple of years ago for work. The idea was
simple: make the tool more accessible for co-workers who use it by
turning it into a web application.

The premise of the tool is extremely simple. It automates the sending
of multiple cURL requests to a WordPress site's xmlrpc.php file with
some extra QOL features.

The purpose of automating the sending of multiple requests is that a
single cURL request may succeed, but the WordPress mobile app and
Jetpack do not make single requests - they make many requests and
need to be in constant communication with that file. A single request
might succeed, but many can fail due to rate limiting, bandwidth or
other hosting limitations. It is impossible to fully diagnose a
possible XML-RPC issue without performing a test like this.

The utility works just fine, but I thought converting it to a web app
would make it easier for folks to use. My first thought was to simply
write a PHP script to make the POST request then return the results
to the client. It probably would have taken all of 30 minutes but I
thought "nah that's too much work for a simple app...." So I thought
I'll just port the Python script into a web app using a Python web
framework. Django is far too much for something like this, so I
started looking at Flask. Even Flask seemed a bit overkill for a
one-file project; thankfully there's Bottle which is a
"micro-framework" similar to Flask.

I decided to spin up a Bottle app which is ridiculously easy and had
a working version of the script running on a web page in about an
hour.

[nopatience]

It worked just fine, but I wasn't sure I liked the idea that the user
would have to wait during the request for the script to complete all
n number of requests before the results page loaded. I also thought
that JavaScript might be a better solution in general: I could just
make the requests via JavaScript and append the results to the DOM as
they came in. I wouldn't even need a server component. It would be so
simple...

[jim_carrey_me_myself_and_irene_ohboy_gif]

The first thing I did was re-write the request using JavaScript's
Fetch API. This took a little while because I needed to refresh on
Async Functions and Promises. I feel like every time I look at
JavaScript it's different . After translating the request to a
JavaScript acceptable request:

let mySite = document.getElementById('siteURL');

    let myRequest = await fetch(mySite, {
        headers: {
            "Content-Type": "text/xml",
            "User-Agent": "Jetpack by WordPress.com",
        },
        body : '<?xml version="1.0"?><methodCall><methodName>demo.sayHello</methodName><params></params></methodCall>',
        method: "POST",
    });

The next step was to wrap it into an Async function for looping:

const MyAsyncFunc = async () => {
event.preventDefault(); //don't reload the page
}

And then wrap the previous code into a loop and append the results to
the DOM.

const MyAsyncFunc = async () => {
event.preventDefault(); //don't reload the page

  for (let i = 0; i < 50; i++){

    let mySite = document.getElementById('siteURL');

    let myRequest = await fetch(mySite, {
        headers: {
            "Content-Type": "text/xml",
            "User-Agent": "Jetpack by WordPress.com",
        },
        body : "<?xml version=1.0?><methodCall><methodName>demo.sayHello</methodName><params></params></methodCall>",
        method: "POST",
    });

    //console.log("Request #" + i + "HTTP Response: " + myRequest.status + " " + myRequest.statusText);

    document.getElementById("results").innerHTML += `<div>${"Request # " + (i + 1) + " | HTTP Response: " + myRequest.status + " " + myRequest.statusText}</div>`;
    }
}

This worked GREAT except:

[lhajv0cyq6u8768ajyhjla]

The request was coming from a different origin than the server, so it
was blocked by CORS which is a security feature imposed by all modern
browsers. From the wiki:

    Cross-origin resource sharing (CORS) is a mechanism that allows
    restricted resources on a web page to be requested from another
    domain outside the domain from which the first resource was
    served.

    A web page may freely embed cross-origin images, stylesheets,
    scripts, iframes, and videos. Certain "cross-domain" requests,
    notably Ajax requests, are forbidden by default by the
    same-origin security policy. CORS defines a way in which a
    browser and server can interact to determine whether it is safe
    to allow the cross-origin request. It allows for more freedom and
    functionality than purely same-origin requests, but is more
    secure than simply allowing all cross-origin requests.

    - Cross-origin resource sharing

Shit. I was never going to be able to be in control of the server
from which the client request originated, since this tool in essence
needs to be able to query any WordPress site. I understand the
security implementations, but was kind of frustrated. I could get
exactly what I needed in one cURL request, now I had to find a way to
work around this. After some discussion on the Handmade Network, I
confirmed that the request could be made from node or another tool on
the server where CORS doesn't exist for the request to the target
WordPress site, and I would control the CORS between the client and
the server.

This quickly became more complicated than it needed to be. I could
have just run the information through the back end and load it into a
new route but that would be just like the Python solution. The idea
was to run the request async and append the results to the DOM.

[screenshot_21]

Of course, the actual request would look something like this:

[screenshot_22]

Except it's even more complicated because you can't just pass
something to the server like that and get it back asynchronously....
you have to set up an API! So I set up a small REST API to receive a
GET request appended to the URL as the URL. Then sent that URL off in
a fetch request, then sent the promise back to the client as the
response data:

app.get("/:url", (req, res, next) => {
    var recievedURL = req.params.url;
    console.log("Got URL: " + recievedURL);
    fetch(recievedURL, {
        headers: {
            "Content-Type": "text/xml",
            "User-Agent": "Jetpack by WordPress.com",
        },
        body : '<?xml version="1.0"?><methodCall><methodName>demo.sayHello</methodName><params></params></methodCall>',
        method: "POST",
        mode: "cors"
    }).then(data => res.send(data));
    console.log("Sent Data");
});

Yeah, it was messy as shit. It was also getting harder and harder to
debug as JavaScript has no timeout functionality I would have had to
wrap that request in another promise. It was all just so needlessly
complicated and frustrating. I kept going back to my command line and
running that single cURL request, getting exactly what I needed, and
getting pissed.

Ultimately I decided that the node app + REST API was just more
trouble than it was worth. Two packages installed and I had a bloaty
node_modules directory with 73 folders and 335 files.

[screenshot_23]I hate node.

Ultimately I said "screw it" and decided the whole API approach for
this one-page app was WAY more trouble than it was worth. I would
have had to maintain the client, server, and node packages because
you know those things have to be updated every 9 minutes or you have
a security vulnerability .

So, 72 hours later and I was back to where I started, with my simple
yet effective one file bottle app. The source is 6kb and can handle
what it needs to, the error logging is verbose and it works how I
expect it to. The tradeoff? People have to wait 1-30 seconds for the
request to complete or timeout, so I added a GIF to help them with
the wait:

[starwars_have_patience_obi_wan]Three days wasted. I should have just
written the PHP script .

My friend & co-worker Brooke helped put some simple styling on it
(also helped with my workout program design) and I think the final
result looks pretty sleek:

  * [screenshot_21-1]
  * [screenshot_22-1]

I also got to do all of the fun systems stuff like remember how
Apache works, update certbot from ACME v1, and play with the virtual
hosts file which is always a good time.

[this-is-great-im-having-the-time-of-my-life]

I spent some serious time on input sanitation and error catching, so
it should be interesting to see how people break it 

Share this:

  * Facebook
  * Twitter
  * LinkedIn
  * More
  * 

  * Email
  * Print
  * 

Like Loading...

Related

[095d60809b78f]

Published by Rudy

I believe in conscious continuous improvement. View all posts by Rudy

Post navigation

Previous Previous post: Free game assets
Next Next post: Draw! is now playable

2 thoughts on "Building an "easy" web application"

Add Comment

 1. [5f2] A. I. Sajib says:
    July 7, 2020 at 2:39 am

    Someday, I'll understand what's going on here. (Jk, I probably
    never will.)

    LikeLiked by 1 person

    Reply
     1. [095] Rudy says:
        July 7, 2020 at 7:15 am

        You will..... or you will be smarter than me and stop after you
        write the bottle app instead of wasting your weekend
        re-writing it in every language/framework then writing a blog
        post only to end up using the bottle app you spent 30 minutes
        writing 

        LikeLiked by 1 person

        Reply

Leave a comment Cancel reply

 [                                             ] 
 [                                             ] 
 [                                             ] 
 [                                             ] 
 [                                             ] 
 [                                             ] 
 [                                             ] 
D[                                             ] 

This site uses Akismet to reduce spam. Learn how your comment data is
processed.

Recent Posts

  * Replacing a Failed SSD in My Dell Optiplex 9020 Homelab Server
  * Journaling, Blogging, and Embracing Imperfection
  * Life @ Automattic: Sysops
  * Global Game Jam 2021
  * 2021: Complete projects, current projects, and beyond!

Follow on Social

  * Twitter
  * Facebook
  * LinkedIn
  * YouTube
  * GitHub

My Tweets

Follow this Site

Enter your email address to follow this blog and receive
notifications of new posts by email. I won't send you anything,
you'll just receive post notifications.

Email Address: [                    ]

Follow

Categories

  * 100 Days of Running
  * Books
  * Crypto
  * diy
  * Dogs
  * Fitness
  * Games
  * Hardware
  * Movies
  * Personal
  * Portfolio
  * Software
  * Sports
  * Technology
  * Travel
  * Uncategorized
  * Work

Archives

  * September 2024 (1)
  * April 2024 (1)
  * August 2023 (1)
  * February 2021 (1)
  * January 2021 (1)
  * November 2020 (2)
  * October 2020 (1)
  * September 2020 (1)
  * August 2020 (2)
  * July 2020 (1)
  * June 2020 (3)
  * April 2020 (3)
  * February 2020 (1)
  * December 2019 (2)
  * November 2019 (1)
  * October 2019 (1)
  * August 2019 (1)
  * May 2019 (3)
  * April 2019 (17)
  * March 2019 (29)
  * February 2019 (27)
  * January 2019 (30)
  * December 2018 (3)
  * October 2018 (1)
  * July 2018 (3)
  * June 2018 (1)
  * December 2017 (3)
  * August 2017 (1)
  * April 2017 (1)
  * August 2016 (1)

 

(c) 2025 Rudy Faile

  * Comment
  * Reblog
  * Subscribe Subscribed
      + [rudyfa] Rudy Faile
        Join 136 other subscribers
        [                    ]
        Sign me up
      + Already have a WordPress.com account? Log in now.
  * 
      + [rudyfa] Rudy Faile
      + Subscribe Subscribed
      + Sign up
      + Log in
      + Copy shortlink
      + Report this content
      + View post in Reader
      + Manage subscriptions
      + Collapse this bar

 

  

Loading Comments...
 
Write a Comment... [                    ]
Email (Required) [                    ] Name (Required)
[                    ] Website [                    ]
[Post Comment]

 
%d

[b]