Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce VueJS, make not-grocy a SPA. #25

Draft
wants to merge 17 commits into
base: master
Choose a base branch
from
Draft

Introduce VueJS, make not-grocy a SPA. #25

wants to merge 17 commits into from

Conversation

hackathi
Copy link
Owner

@hackathi hackathi commented Jul 2, 2021

This is a very early draft.

  • Installed vue 3, primeVue, typescript
  • Working rollup build
  • linting and formatting (old rules) in VS Code
  • basic template, not loading old views
  • vue-router
  • vue-i18n
  • nightmode
  • load legacy views
  • testing infrastructure
  • tests pure-vue views
  • Get GrocySettings from API call
  • vuex

fixes #3
fixes #6

@hackathi hackathi marked this pull request as draft July 2, 2021 21:07
@@ -0,0 +1,6 @@
import { openBlock as _openBlock, createBlock as _createBlock } from "vue";

export function render(_ctx, _cache, $props, $setup, $data, $options)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like a build artifact.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. At some point I completely messed up the rollup config, and that's that. Also, eslint breaks in the vue-rollup config. I have no idea why. Linting in VS Code or with the command line works, so I am not concerned.

@hackathi
Copy link
Owner Author

hackathi commented Jul 3, 2021

image

Current preview. I settled for a green look with a few colorful accents here and there. Still pretty much WIP. Also I made a new logo. It's very simplistic atm and I need to do some better versions of it (or pay a friend of mine to do it for me).

Also stickers. I want stickers now.

Night mode:
image

@hackathi
Copy link
Owner Author

hackathi commented Jul 3, 2021

image

UserSettings Panel. A visual improvement for sure.

@hackathi
Copy link
Owner Author

hackathi commented Jul 3, 2021

Next up: i18n.

hackathi added 2 commits July 3, 2021 19:04
I have commited the script I used to convert the gettext files.
@hackathi
Copy link
Owner Author

hackathi commented Jul 3, 2021

i18n is here again! Also static extraction with a makefile target make extract.

@mohe2015
Copy link
Contributor

mohe2015 commented Jul 3, 2021

diff --git a/js/legacy/helpers/qrcode.js b/js/legacy/helpers/qrcode.js
index b6168f21..aa884a34 100644
--- a/js/legacy/helpers/qrcode.js
+++ b/js/legacy/helpers/qrcode.js
@@ -1,5 +1,5 @@
 // bwipjs is broken, needs to be explicitly imported.
-import bwipJs from '../../node_modules/bwip-js/dist/bwip-js.mjs';
+import bwipJs from '../../../node_modules/bwip-js/dist/bwip-js.mjs';
 
 function QrCodeImgHtml(text) 
 {
diff --git a/js/legacy/viewjs/shoppinglist.js b/js/legacy/viewjs/shoppinglist.js
index e14fb8ef..fe09552a 100644
--- a/js/legacy/viewjs/shoppinglist.js
+++ b/js/legacy/viewjs/shoppinglist.js
@@ -2,7 +2,7 @@ import { animateCSS } from '../helpers/extensions';
 import { __t, U } from '../lib/legacy'; //import { $ } from 'jquery'; // this needs to be explicitly imported for some reason,
 // otherwise rollup complains.
 
-import bwipjs from '../../node_modules/bwip-js/dist/bwip-js.mjs';
+import bwipjs from '../../../node_modules/bwip-js/dist/bwip-js.mjs';
 import { WindowMessageBag } from '../helpers/messagebag';
 
 function shoppinglistView(Grocy, scope = null)

And also

<br />
<b>Fatal error</b>:  Uncaught Error: Call to a member function toJsonString() on null in /home/moritz/Documents/not-grocy/php/Services/LocalizationService.php:66
Stack trace:
#0 /home/moritz/Documents/not-grocy/php/Controllers/BaseController.php(140): Grocy\Services\LocalizationService-&gt;GetPoAsJsonString()
#1 /home/moritz/Documents/not-grocy/php/Controllers/BaseController.php(220): Grocy\Controllers\BaseController-&gt;render(Object(Slim\Http\ServerRequest), Object(Slim\Http\Response), 'errors/500', Array)
#2 /home/moritz/Documents/not-grocy/php/Controllers/ExceptionController.php(77): Grocy\Controllers\BaseController-&gt;renderPage(Object(Slim\Http\ServerRequest), Object(Slim\Http\Response), 'errors/500', Array)
#3 /home/moritz/Documents/not-grocy/vendor/slim/slim/Slim/Middleware/ErrorMiddleware.php(127): Grocy\Controllers\ExceptionController-&gt;__invoke(Object(Slim\Http\ServerRequest), Object(PDOException), true, false, false)
#4 /home/moritz/Documents/not-grocy/vendor/slim/slim/Slim/Middleware/ErrorMiddleware.php(109): Slim\Middleware\ in <b>/home/moritz/Documents/not-grocy/php/Services/LocalizationService.php</b> on line <b>66</b><br />

again. So can't check it out yet :(

@hackathi
Copy link
Owner Author

hackathi commented Jul 3, 2021

I've seen the bwip-js errors, but have chosen to ignore them for the time being 😅 I'll push a fix in a sec.

You know what? I'll just disable that json generation. It's completely useless for vue anyways. Can't throw if it ain't there.

@mohe2015
Copy link
Contributor

mohe2015 commented Jul 3, 2021

Yay CI worked! (SQLSTATE[HY000]: General error: 1 no such table: users) because migrations don't run any more when opening the index page. This is probably expected as I assume you just replaced the index page with the vue page.

@hackathi
Copy link
Owner Author

hackathi commented Jul 3, 2021

Hmmm... My PR shouldn't mess with migration.

public function Root(\Psr\Http\Message\ServerRequestInterface $request, \Psr\Http\Message\ResponseInterface $response, array $args)
{
    // Schema migration is done here
    $databaseMigrationService = DatabaseMigrationService::getInstance();
    $databaseMigrationService->MigrateDatabase();
    
    if (GROCY_MODE === 'dev' || GROCY_MODE === 'demo' || GROCY_MODE === 'prerelease')
    {
	    $demoDataGeneratorService = DemoDataGeneratorService::getInstance();
	    $demoDataGeneratorService->PopulateDemoData();
    }
    
    // yolo, this is a SPA now.
    return $this->render($request, $response, 'vue');
    //return $response->withRedirect($this->AppContainer->get('UrlManager')->ConstructUrl($this->GetEntryPageRelative()));
}

No Idea why the migrations don't run.

@mohe2015
Copy link
Contributor

mohe2015 commented Jul 3, 2021

diff --git a/php/Controllers/SystemController.php b/php/Controllers/SystemController.php
index 39b7b702..8cfb582b 100644
--- a/php/Controllers/SystemController.php
+++ b/php/Controllers/SystemController.php
@@ -4,6 +4,7 @@ namespace Grocy\Controllers;
 
 use Grocy\Services\DatabaseMigrationService;
 use Grocy\Services\DemoDataGeneratorService;
+use Grocy\Services\SessionService;
 
 class SystemController extends BaseController
 {

otherwise it searches it in Grocy\Controllers as far as I understand

@hackathi
Copy link
Owner Author

hackathi commented Jul 3, 2021

I will be forever grateful when the day comes and I can rip out that DBAL.

@mohe2015
Copy link
Contributor

mohe2015 commented Jul 3, 2021

I really like it - also tested i18n by switching languages manually and it works really well.

I was wondering if adding the frontend target to the build target would make sense:

diff --git a/Makefile b/Makefile
index eecec615..c0980a45 100644
--- a/Makefile
+++ b/Makefile
@@ -83,7 +83,7 @@ help:
 
 # Install all dependencies, then build the public/ folder.
 .PHONY=build
-build: vendor js css public/js/locales/grocy/en.json resources
+build: vendor js frontend css public/js/locales/grocy/en.json resources
 
 .PHONY=minify
 minify: build

Also I have some issues with make watch. Sass creates empty CSS files although building which does basically the same thing works. I will look into this tomorrow further but maybe you also have experienced that.

@hackathi
Copy link
Owner Author

hackathi commented Jul 3, 2021

I was wondering if adding the frontend target to the build target would make sense:

Yes, of course. I missed that.

Also I have some issues with make watch. Sass creates empty CSS files although building which does basically the same thing works. I will look into this tomorrow further but maybe you also have experienced that.

This could also be postcss failing. postcss in watch mode can't replace so sass builds to a temp file, which postcss watches and builds from there.

@hackathi
Copy link
Owner Author

hackathi commented Jul 4, 2021

We have now reached "if it's a button that is not a link to another view, it does something" stage.

@mohe2015
Copy link
Contributor

mohe2015 commented Jul 4, 2021

it does something works now - unfortunately the does something things don't have text anymore. Do you know why because the menu on the left still has text.
image

@hackathi
Copy link
Owner Author

hackathi commented Jul 4, 2021

You need to re-copy the locales (... forgot to add it to the makefile again -.-) and everything should work (tm). make resources should do.

Or i f'ed something else up. classic works-on-my-machine.

I'm almost ready to push a new set of changes, that should fix it, if nothing else.

@hackathi
Copy link
Owner Author

hackathi commented Jul 4, 2021

2a5ff20 should fix the issues.

Also I've taken the liberty and created a basic stock overview page:

image

The Stock table is incomplete and needs to be wrapped in a component, but still. I do like. Stockoverview now loads in ~350ms on my machine. Yay.

@hackathi
Copy link
Owner Author

hackathi commented Jul 4, 2021

Next up: create one of the other, easier pages so I can tick vue-router off my list, then off to legacy view loading.

Once we got every "stage" worked out once I guess we could merge and do individual pages / components per PR.

@hackathi
Copy link
Owner Author

hackathi commented Jul 4, 2021

The dot menu is there btw, It's just a context menu now:

image

@mohe2015
Copy link
Contributor

mohe2015 commented Jul 4, 2021

I'm pretty sure something is broken as most translations load but not all.
I tried:

make resources
make extract
make build
make frontend
make run

and it didn't work.
image
for reference of whats working and what not.

@hackathi
Copy link
Owner Author

hackathi commented Jul 5, 2021

I think I got it sorted now. The option to format fallback strings from the vue-i18n docs wasn't the option the plugin actually uses. So it didn't format them.

Regarding the empty strings: this occurs whenever a translation is missing, but its key is present in the language dictionary. At the moment, making the languages is just a simple cp $< $@; I think we need to do some processing there as to not include empty strings (i.e. missing translations).

@hackathi
Copy link
Owner Author

hackathi commented Jul 5, 2021

Also, the API is incredibly dirty with the types it returns. For now, I think we should fix that in the frontend by properly abstracting the loading or something (the processing I do in the stockoverview page at the moment is far from ideal).

I don't want to touch existing API endpoints if I don't have to. Making a better typed v2 API is a thing for the future.

@hackathi
Copy link
Owner Author

hackathi commented Jul 6, 2021

image

I can cross vue-router off my list. Consume / Put on Shoppinglist / Fullscreen / Print doesn't work yet, Filters / Sorting isn't there, adding of recipes doesn't work.

But I can click through the desired servings without reloading the page each time I click on the button, which is obviously way more important.

@mohe2015
Copy link
Contributor

mohe2015 commented Jul 7, 2021

I can cross vue-router off my list. Consume / Put on Shoppinglist / Fullscreen / Print doesn't work yet, Filters / Sorting isn't there, adding of recipes doesn't work.

But I can click through the desired servings without reloading the page each time I click on the button, which is obviously way more important.

Is this pushed?

@hackathi
Copy link
Owner Author

hackathi commented Jul 8, 2021

Not yet, there were a few Bugs I needed to Iron out. I'll push what I have and fix the history later.

@hackathi
Copy link
Owner Author

hackathi commented Jul 8, 2021

Regarding all the cleanXXX() functions I introduced – my best guess why they are needed is because SQLite doesn't care about types and happily inserts strings into INTEGER columns; which is what happening everywhere.

The more I look at the backend and the database, the more I see myself nuking it once the frontend is done and replacing it with something sane and strictly typed (I'm thinking .net core API?). Not running PHP has some benefits with the obvious drawback that it's less accessible to install for people.

python/django would also be an option, if we typehint everything from the ground up.

@mohe2015
Copy link
Contributor

mohe2015 commented Jul 8, 2021

Regarding all the cleanXXX() functions I introduced – my best guess why they are needed is because SQLite doesn't care about types and happily inserts strings into INTEGER columns; which is what happening everywhere.

Yeah SQLite doesn't have strict types.

The more I look at the backend and the database, the more I see myself nuking it once the frontend is done and replacing it with something sane and strictly typed (I'm thinking .net core API?). Not running PHP has some benefits with the obvious drawback that it's less accessible to install for people.

.net on linux - omg </microsoft-bashing>

python/django would also be an option, if we typehint everything from the ground up.

But I want to run this on a pi ^^ </python-bashing>

I don't know my time commitment yet but I'm thinking about writing it in Rust with actix-web, diesel, https://github.com/sfackler/r2d2 just for fun.
We probably need to handle migration from the old database schema - don't we?
If I would start this I will probably do this in a separate repository.
The server side API is expected to not change too much (except maybe for correct types)?

@mohe2015
Copy link
Contributor

mohe2015 commented Jul 8, 2021

If we change the JSON types to integers and doubles we have to take care that we don't miscalculate things because of precision errors. E.g. 1.7 can't be exactly represented in binary: (1.7).toPrecision(100) results in "1.699999999999999955591079014993738383054733276367187500000000000000000000000000000000000000000000000".
This shouldn't really matter for our use case as we can use rounding but this has to be kept in mind.

@mohe2015
Copy link
Contributor

mohe2015 commented Jul 8, 2021

https://github.com/mohe2015/not-grocy-server is the proof of concept (not cleaned up and not even fully finished the first api endpoint). The backend code, schema, migrations, sql views etc. of grocy is a mess so I agree with you that we (I) should probably first document it and then decide how to migrate and improve it. Also understanding the queries is impossible without knowing the schema exactly (e.g. https://github.com/mistressofjellyfish/not-grocy/blob/246c7fcb64d63caf1a4a1e8dafb0308d9fa4264b/php/Migrations/0122.sql).

I don't know how much we want to change the frontend-backend integration but I think it could be possible many of the queries can be simplified and unified.

P.S.: Rust is probably not the best language but I was interested in how easy/hard it would be.

@hackathi
Copy link
Owner Author

hackathi commented Jul 8, 2021

We probably need to handle migration from the old database schema - don't we?

yes; but I‘m fine with one way conversion; just make it reasonably easy for other people to write the other way if they want (i.e. documentation of the schemas philosophy). And it needs to be compatible to at least sqlite and PostgreSQL. If it runs on MySQL, it‘s a plus, but I don‘t care if it does.

If we change the JSON types to integers and doubles we have to take care that we don't miscalculate things because of precision errors.

As it currently stands, the php code coerces the types whenever it feels like it. The conversion is necessary on the frontend anyways to apply number formatting rules, so I don‘t see any new problems there. The database and all number calculations are done on floating point anything anyways, so rounding errors have always been there. That’s okay for weights and stuff where you can‘t really measure .0001 of a difference in most cases, but it‘s bad for money related things.

https://github.com/mohe2015/not-grocy-server is the proof of concept (not cleaned up and not even fully finished the first api endpoint).

I really, really like the idea, but I really do wonder if rust is a particularly good fit here. It’s fast, obviously, but I‘m absolutely not familiar with the tooling. Not the compiler toolchain, but migrations, ORM, deployment… If you have some links to something that’s more or less considered canon, they‘d be much appreciated.

My reasoning for python or .net core was that both environments are well understood and have good tooling that‘s very accessible to new people. There are many guides for deployment and a huge stack exchange community for both. Do you know how the situation is for rust? Especially for actix-web I vaguely remember some drama a while ago, but don‘t recall the details.

Personally, I‘d rather use boring tech with great tooling than fancy tech with immature tooling. I can‘t tell how the situation is with rust, but I would rather go a more conservative route, even if it is only to keep the pool of potential contributors larger.

@mohe2015
Copy link
Contributor

mohe2015 commented Jul 8, 2021

I really, really like the idea, but I really do wonder if rust is a particularly good fit here. It’s fast, obviously, but I‘m absolutely not familiar with the tooling. Not the compiler toolchain, but migrations, ORM, deployment… If you have some links to something that’s more or less considered canon, they‘d be much appreciated.
Personally, I‘d rather use boring tech with great tooling than fancy tech with immature tooling. I can‘t tell how the situation is with rust, but I would rather go a more conservative route, even if it is only to keep the pool of potential contributors larger.

I'm pretty sure it's not - not necessarily because the tooling is bad but rather because way fewer people know it and some concepts are pretty different from traditional languages. I think I know quite some languages (at least partially) and I would consider Rust to be one of the harder ones as it doesn't blindly adopt the concepts of all the others. We probably don't need many of the more advanced features like lifetimes but basic knowledge about borrowing and traits is probably still needed.
I chose the most popular frameworks and I would consider the actix-web, diesel and r2d2 documentation and tools to be pretty mature and well working. So I would say most problems would be being new to Rust in general and not these tools.

I would consider the official documentation at https://actix.rs/docs/, https://diesel.rs/guides/getting-started, https://diesel.rs/guides/ to be pretty good. The error messages of the Rust compiler are usually also better than what I saw in other compilers.
For Rust in general for a quick start there is https://doc.rust-lang.org/stable/rust-by-example/index.html or alternatively https://doc.rust-lang.org/book/title-page.html.

I don't know how we should proceed with that. I could imagine that most of the APi is pretty easy to implement and the main point is to have an ORM and types. I didn't look at the PHP code yet but the calendar could be harder in Rust (didn't look for libraries or how hard this is yet).

Another (major depending on who you ask) disadvantage is the cpu power needed for compiling Rust programs. If you have a low power pc this would be really annoying.

Deployment should be pretty standard (I don't know how you deploy django nor .net core) but off the top of my head you could create a systemd-service and either directly expose the http endpoint or proxy it through nginx or so. We could probably provide binaries for the common platforms but otherwise it's a cargo build (with rustup installed) away.

The next thing I would probably look into is the existing schema etc. and this will benefit us anyways.

Considering the amount of contributors there where in the past Rust may be an option (in the sense it won't be worse) but I'm pretty sure we will have fewer contributors. It may also be an opportunity for some to get to know Rust though.

P.S.: I don't know django (python really) nor .net core so I would need to learn that (I have programmed C# in the past) but I could learn that of course.

@hackathi
Copy link
Owner Author

hackathi commented Jul 8, 2021

Considering the amount of contributors there where in the past Rust may be an option (in the sense it won't be worse)

Yes. But really I think that this is mostly attributed to the nonexisting contributing guidelines, automation, CI, testing, the overall bad code and its many inherent problems, and how people in the past were handled. I saw like two? real efforts to have MySQL shut down in pretty much the same way my refactoring has been shut down. It doesn't really surprise me that nobody wants to touch that thing if there is no easy way to do so, documentation is nowhere to be found, and there is no way of knowing the side effects of your work.

I do want to do that differently for not-grocy, and I prefer developer efficiency and easy onbording over the last bit of performance. For a typical home installation server response time give or take a few ms doesn't matter. Anything that just starts and gets reverse proxied connections from, say, nginx, will probably fulfill that.

So now we have a question of usability (for end users), ease of contribution and ease of plugin extendability (because, really, not everything can or should be handled within core not-grocy). And for none of these I see a case for rust. The build process takes ages and huge amounts of RAM and results in a large, statically linked blob. I think the learning curve for rust is pretty steep, so it doesn't score well on ease of contribution either (in all reality: I wouldn't feel comfortable contributing to an actual rust codebase, even though I have a pretty good understanding of the inner workings of the language). I don't know about plugin extensibility, but I suspect that the compiler checked guarantees are gone because the interface needs to be unsafe.

So, keeping this in mind, I really don't see the case for rust, even though it's absolutely fascinating tech.

From the standpoint of making it easy for contributors, there are basically three choices: node.js / javascript backend, php (but saner) or python. I do like .net core and have some relationship with mono and the .net ecosystem, so I might be biased there. But I also know that it is turning some people off. So my final tradeoff would be python3/django which makes things easier for more contributors but is not statically typed or c# .net core MVC API with Entity Framework which makes all calculations that need to be done on specific types much easier. Or stick to php, but I really, really, really dislike practically all php ORMs.

But in all reality: Before attempting to fundamentally change the backend, we need to document and understand the current database schema, and all API endpoints. I really want to avoid changing existing API endpoints while re-doing the frontend, and even if we redo the core at some point, I don't really want to throw away the current API all at once. At some point it would be sensible to re-do some of the endpoints, but that will include versioning the new api to /api/v2/<endpoints>. So, I think we should open a discussion outside of the PR of what to do with the backend (it's a bit off topic here), because I'm sure there are more things to consider that I don't think of right now.

Speaking of the frontend; I still have "loading legacy views" unresolved. And I am really considering just rewriting them all. I realized that the things that bother me are all in the more complex views and boil down to nonsensical page reloads that just shouldn't be there in the first place. I wouldn't want to legacy-load them anyway.

I'll finish up on the recipe pages to make adding and editing work, then I'll do some testing. Once this is set up, I'm thinking about merging, and just chugging away a page per day. At the moment, I'm mostly doing bootstrapping anyways, which will get less the more I progress.

@mohe2015
Copy link
Contributor

mohe2015 commented Jul 9, 2021

I agree that all in all Rust is not the best for an official backend. I don't know if I will work on it anyways for myself as I haven't done a web server project in rust yet. You could argue this is wasted work but otherwise I would probably program something else so I think it's fine. I'm pretty sure it won't get plugin support though.

statically linked blob

Small correction: Rust doesn't produce statically linked binaries by default - I think that's Go.
Edit: Quickly googling it seems like only libc is dynamically linked - doesn't matter though as all other crates would only be used once anyways so it would only help if you have multiple rust projects with the same dependencies

@hackathi
Copy link
Owner Author

hackathi commented Jul 9, 2021

You could argue this is wasted work but otherwise I would probably program something else so I think it's fine.

If you have fun with it, go for it! I often times need to re-implement things to really understand them. And most of the time I actually learn a few things doing that. So – wasted in the sense of "not beneficial for not-grocy"? Probably, but better understanding what is there is. And if you learn something on the way it's beneficial for yourself, and that is way more important than any considerations for a random open source project.

@mohe2015
Copy link
Contributor

mohe2015 commented Jul 9, 2021

Speaking of the frontend; I still have "loading legacy views" unresolved. And I am really considering just rewriting them all. I realized that the things that bother me are all in the more complex views and boil down to nonsensical page reloads that just shouldn't be there in the first place. I wouldn't want to legacy-load them anyway.

I can imagine that this is more effective.

I'll finish up on the recipe pages to make adding and editing work, then I'll do some testing. Once this is set up, I'm thinking about merging, and just chugging away a page per day. At the moment, I'm mostly doing bootstrapping anyways, which will get less the more I progress.

👍

@mohe2015 mohe2015 mentioned this pull request Jul 9, 2021
hackathi added 2 commits July 9, 2021 19:17
Also bits & pieces of Product amount picker, recipe edit form
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Move frontend translation to vue-i18n Move not-grocy to a SPA using vue3
2 participants