To get the general idea, you can take a look at our presentation for this project.
NOTE: the most current progress is on Ats2 branch.
This project is organized as such:
Scraper(Class library) - has actual scraper implementationsScraperFunction- an Azure Function which runs regularly and puts scraped contents into a database; it's hosted on Azure.CategoryMap- contains commoncategory treecategories and mapper classes which catagories from other stores are mapped onto.Models- contains classes used through out the project. NOTE:BlazeCartuses different classes to represent the same data, such asItemandCategory.DB(Entity Framework) - contains code-first DBContextsApi(ASP.NET Core Web API) - REST API for accessing scraped contents from DBBlazeCart(MAUI) - the UI, mobile (Android) appCommon- contains various helper extention methodsTests- it's... tests
The client relies on a working instance of Api, which relies on DB, which is populated by ScraperFunction; see Untitled Diagram.drawio for clarity.
Updates to DB schema are done with dotnet ef cli tool.
Steps:
dotnet ef migrations add <Migration name>- generates migration(s) from changesdotnet ef database update- actually updates the database
Migrations are needed then properties of classes Entity, Category, Item have been changed, added or removed.
To run all tests: dotnet test to test.
To run a specific test use --filter, example: dotnet test --filter "FullyQualifiedName=Tests1.Scraper.BarboraScarperTest.DuplicateTest"
NOTE: CategoryMap/Implementations tests may be incorrect thus may not pass even if the code is correct.
In theory iOS is supported, but hasn't been tested since it was primarily developed on Android. The app is based on mvvm .
Click here to see the design.
This project uses DevExpress, you'll need to obtain feed url from them and add it to your Nuget sources.
Use Visual Studio or cli and then either: connect a physical device via adb or use an emulator.
- MAUI check
- MVVM and MAUI
- Data Caching
- JSON to C# classes
- Curl to C# code converter
- Cron expression generator
If you have run into problems make consult with this checklist:
- DB connection strings are added in configs (appsettings.json, secrets.json).
- Dotnet and dependency versions match (this project targets
net6.0).
Api may take a few minutes to start up, it may send 503 response method while it does.
-
You may need to fetch app-settings using func.
Run
func azure functionapp fetch-app-settings <function-name> -
If you get this:
[TIMESTAMP] Executed 'ScraperFunction' (Failed, Id=[...], Duration=80ms) [TIMESTAMP] System.Private.Uri: Value cannot be null. (Parameter 'uriString').This may mean that (Consult Google first for most common causes)
Urifield isnullin someCategoryorItem, because some part of code will try to deep clone it. [NOTE: MAY BE OUTDATED INFO]
As many student projects and assignments, this one was done in quick spirts before a deadline and then almost abandomed for weeks. As a result many features were left half-baked, code quality may have suffered.