I want to improve Prince of Persia Sands of Time PC's gamepad controls with this program but I checked the webpage and the language is in Japanese:

https://sites.google.com/site/0dd14lab/xinput-plus

To me that makes it looks sketchy because I have seen many people recommending this and never mentioned why its in Japanese. I worry that if I download the program it will be in Japanese.

And I worry about a potential virus/malware infection that might come with the download.

It's in Japanese not because it's sketchy but because the dev is Japanese. It looks "clean" and there's a tutorial in English for it here (including what to click on to change language):-
https://www.play-old-pc-games.com/compatibility-tools/xinput-plus-tutorial/

Did you download yourself to see if its ok?

I haven't tested it myself, but it's one of those utilities where if it were malware, then a lot of respected sites wouldn't be linking to it or creating lengthy tutorials on how to use it (nor would Google still be hosting it). You could always download it and use a virus scanner before installing.

I have done so with my Avira Antivirus, scanned it, and its clean.

I think you're safe then. It's not the only popular utility with a Japanese author, eg, CrystalDiskInfo & CrystalDiskMark are coded by Noriyuki Miyazaki, and seeing the Japanese character set in early releases caused some people at the time to question "is it safe" too.

That doesn't mean it's safe.

Look at this for instance.
Normal malwarebytes didn't pick this up but the pro version did. Other scanners including windows didn't either.
The situation is strange and it doesn't help that paypal and the dev played fast and loose with peoples info. A "gary" seems to be in control after reports of the death* of the dev.

https://old.reddit.com/r/pcgaming/comments/ca4y0z/psa_about_pinnacle_game_profiler/

*https://www.pearsonfh.com/obituary/robert-kulanko

Should I delete the thing then?

I haven't tried XinputPlus if that's what you mean. It's probably ok

If you have downloaded pinnacle from its main site then I would delete that.

I just downloaded Xinputplus as is.