Add README.md

.gitmodules

@@ -0,0 +1,6 @@
+[submodule "hw_html/xmlParser"]
+	path = hw_html/xmlParser
+	url = https://github.com/lookup69/xmlParser
+[submodule "hw_html/CTML"]
+	path = hw_html/CTML
+	url = https://github.com/tinfoilboy/CTML

README.md

@@ -0,0 +1,46 @@
+# A9 free
+## What is the A9
+The A9 is based on a system-on-a-chip namely RDA8955, made by RDA Technologies.
+It is a very powerful and versatile GSM/GPRS-capable microcontroller that
+was definitely conceived for mobile use, and probably used by various mobile
+phone manufacturers during the 2000's (personal opinion). A GPS-enabled version
+is also available, called A9G.
+
+## What is this project
+The A9 currently depends on a set of proprietary tools and firmware blobs that
+have been left unmantained since 2018 by its manufacturer, Ai-Thinker.
+Therefore, this project aims to maintain the A9 and A9G by creating a free
+(as in freedom) version developed by the community. The A9 and A9G have great
+potential as IoT devices or even to build low-spec smartphones entirely based
+in free software.
+
+## What has been developed so far
+Currently, no free firmware has been developed yet. The project is on a very
+initial phase where we need to know as much as possible about the system
+(e.g.: reverse-engineering) so we can start developing free software for it.
+`doc/notes.txt` contains a growing amount of information about what has been
+found so far and what still needs to be studied in further detail.
+
+This is a list of ToDo things for this project:
+
+ - [x] De-compile firmware blobs using an automated tool e.g.: Ghidra.
+ - [ ] Understand how the flashing process works.
+ - [ ] Write minimal working example that runs on the hardware.
+ - [ ] Write sample applications that interact with the hardware peripherals.
+ - [ ] Port newlib and FreeRTOS to replace existing custom implementations.
+ - [ ] Write a free bootloader that allows uploading application code.
+ - [ ] Implement a free GSM/GPRS stack.
+ - [ ] Implement a free GPS stack.
+
+## How to contribute
+This is a very ambitious project that definitely takes a lot of time and
+resources. I am currently alone at this, and it is impossible to achieve all
+these targets without your support. So I encourage anyone interested in this
+project to join and help the project to become a reality. I think the A9 has
+a lot of potential that the community can benefit from.
+
+So please review `doc/notes.txt`, give your feedback, provide new information
+and help development that allows us to use the A9 with free software only.
+Despite being based on closed-source binaries, the repositores provided by
+Ai-Thinker already include lots of documentation that will surely help us in
+knowing about the inner details on the A9 and the binaries themselves.

doc/notes.txt

@@ -1,4 +1,73 @@
-Notes regarding memory mapping
+General notes
+=============
+The A9 is based on a system-on-a-chip namely RDA8955, made by RDA Technologies.
+It is a very powerful and versatile GSM/GPRS-capable microcontroller that
+was definitely conceived for mobile use, and probably used by various mobile
+phone manufacturers during the 2000's (personal opinion).
+
+Ai-Thinker sells a variant with GPS called the A9G. According to
+https://blog.zakkemble.net/remote-mail-notifier-and-gps-tracker , the A9G uses
+a GK9501 GPS receiver.
+
+Ai-Thinker distributes two packages for the A9:
+    - GPRS_C_SDK (https://github.com/ai-thinker-open/gprs_c_sdk): contains
+    .sh/.bat scripts to build user projects and demos. It also includes source
+    code for some libraries (cjson, vlgl and aliyun, among others) and header
+    files for interfacing their closed-source firmware (more on this later).
+
+    - CSDTK (official link dead, mirror https://github.com/pulkin/csdtk42-linux):
+    contains the binaries of modified versions of the GNU toolchain (gcc and
+    binutils), a bunch of tools used to communicate with the microcontrollers
+    namely "cooltools" and lots of configuration files, some of without clear
+    purpose.
+
+Ai-Thinker seems to have stopped development of their repositories since 2018,
+where the latest version was released, except from some minor commits. So this
+project aims to keep maintaining the A9/A9G by creating a free (as in freedom)
+version of these repositories developed by the community.
+
+On the other hand, since Ai-Thinker are only distributing binaries of the GNU
+toolchain, this means they are violating the GNU General Public License v3 by
+not distribuing the source code. This has already been reported to Ai-Thinker on
+Github (https://github.com/Ai-Thinker-Open/GPRS_C_SDK/issues/431) and e-mail
+both to Ai-Thinker themselves and the Free Software Foundation, with no response
+so far.
+
+Two versions of the toolchain are distributed: mips-rda-elf and mips-elf. The
+former relies on versions of gcc and binutils released in 2017 and the latter
+from 2014. It is currently unclear the differences between both toolchains,
+although GPRS_C_SDK uses mips-elf to build a project instead of mips-rda-elf.
+
+Despite Ai-Thinker distributing so many resources, the real deal on the A9 is a
+debug/release pair of closed-source firmware blobs that contain the GSM/GPRS
+and GPS stacks, the bootloader, the C standard library implementation and the
+implementation of all the libraries that are provided to the user. The binaries
+are located at GPRS_C_SDK/platform/csdk/[debug/release]. According to
+mips-elf-size, the .text section of the debug version of SW_V2129_csdk.elf is
+1969176 bytes (around 1.88 MiB), so there is a lot going on in there.
+
+Fortunately for us, the GPRS_C_SDK repository and also the blobs are under the
+MIT license, which gives us freedom to reverse-engineer them. What's more, the
+original authors compiled these .elf files with full debugging information, so
+they can be de-compiled using a reverse-engineering tool such as NSA's Ghidra
+while keeping the original symbol names.
+
+Using this approach, we see open-source libraries such as mbedtls and lwip have
+been used. A major drawback of the closed-source binary is the fact all unused
+code cannot be optimized away by the linker, so if the A9 contains 4 MiB of
+flash memory that leaves the user with 2.12 MiB. It could be worse, but it could
+also be better.
+
+Ghidra does a decent job at de-compiling the binaries, but sometimes it does
+not figure out the names of local variables and shows many "Could not recover
+jumptable at 0x88010a48. Too many branches" warning messages at the end of a
+function. Also, while Ghidra allows exporting to a C source file, it apparently
+only allows exporting as a *single* C source file, which might be cumbersome
+for such a big project (> 14 MiB, > 530,000 lines). So it is a good idea to
+compare the de-compiled code against the available documentation (more on this
+later).
+
+Memory mapping
 ==============================
 build/app/cust.ld (modified by build.sh from
                    gprs_c_sdk/platform/compilation.cust.ld)
@@ -53,8 +122,8 @@ According to the documentation, the A9 features 4 MiB flash memory, and
 0x8800_0000 - 0x8100_0000 > 4 MiB, so could they be two different flash chips
 mapped at different addresses?
 
-rsoft: kuseg is the user segment, while the kernel segments are only accessible
-in kernel mode and differ in caching and translation
+"rsoft: kuseg is the user segment, while the kernel segments are only accessible
+in kernel mode and differ in caching and translation"
 
 Compiling and linking a simple application returns undefined references to:
     __stack
@@ -162,12 +231,13 @@ LOD combination
 csdtk and gprs_c_sdk were designed around a set of proprietary tools, file
 formats and communication protocols.
     - coolwatcher and coolhost would be roughly equivalent to OpenOCD or
-    avrdude.
-    - .lod files are very similar to .hex files, although poorly designed.
+    avrdude, but GUI-based.
+    - .lod files are very similar to .hex files, although IMHO poorly designed.
     More on this later.
     - HST (a seemingly plain old UART despite its fancy name) is used instead
-    of JTAG/SWD.  I am unsure of how source-level debugging is accomplished,
-    despite coolwatcher having an interface to mips-elf-insight.
+    of JTAG, SWD or any other kind of debug interface. However, coolwatcher
+    features mips-elf-insight, so it might be using the HST for source-level
+    debugging.
 
 The .lod file format includes the following information:
     - Optional fields prefixed by '#' (maybe comments?). This is an example of
@@ -231,3 +301,72 @@ two parameters are specified:
     - Null-terminated string "L1_SYNCH_NOT_FOUND"
     - 32-bit integer (0x%x) with value 0x0000006a
     - Unknown 32-bit integer with value 00 0a 80 00
+
+Hardware registers
+==================
+The csdtk package includes a great deal of information about low-level
+registers in XML format (sometimes .xmd extension is used) which could be
+easily parsed and laid out in a more readable format e.g.: HTML. A WIP
+application is being developed for that very same purpose under the hw_html
+directory.
+
+These files are located on csdtk/cooltools/chipgen/Modem2G/toolpool/map/8809,
+where 8955_hard.xml describes low-level registers and 8955_soft_pkg.xmd
+describes software structures and enums. Even C code is embedded into these
+XML files, usually under the <cjoker> tag, and containing macro definitions and
+function declarations. Moreover, this documentation even also includes comments
+on how these low-level registers work.
+
+coolwatcher apparently uses this information for its "Register Viewer" (which
+can be accessed from the "Tools" menu), showing the layout of all low-level
+registers, as well as their addresses and real-time values.
+
+According to this, there is a rather lengthy list of peripherals featured on
+the A9/RDA8955:
+
+- ABB
+- AIF
+- BB
+- BCPU
+- CALENDAR
+- CAMERA
+- CHOLK
+- CIPHER
+- CORDIC
+- CS0/CS1
+- DEBUG_HOST/DEBUG_UART
+- DMA
+- EXCOR
+- FMD
+- GOUDA
+- GPIO
+- I2C
+- IOMUX
+- ITLV
+- KEYPAD
+- PAGE_SPY
+- PMU
+- PWM
+- RF
+- SCI
+- SDMMC
+- SEG_SCAN
+- SPI1/2/3
+- SPI_FLASH
+- TCU
+- TIMER
+- UART
+- USBC
+- VITAC
+
+It is currently unknown what some of these peripherals are used for, and some
+of them are not even mentioned on the official specifications e.g.: CAMERA.
+On the other hand, the same website states 2 SPI interfaces are available,
+although as shown above the register viewer listed SPI1/2/3.
+
+Dual CPU?
+=========
+BCPU and XCPU are mentioned in various places, which suggests a dual-CPU design.
+However, https://ai-thinker-open.github.io/GPRS_C_SDK_DOC/en/hardware/a9.html
+only mentions "RDA 32 bit RISC core, frequency up to 312MHz, with 4k instruction
+cache, 4k data cache", so it is unknown whether two CPUs are actually present.

hw_html/CMakeLists.txt

@@ -0,0 +1,11 @@
+cmake_minimum_required(VERSION 3.0)
+project(hw_html CXX)
+add_definitions(-g)
+add_executable(${PROJECT_NAME} html.cpp main.cpp mcubase.cpp bit.cpp
+    mcureg.cpp)
+add_library(xmlParser xmlParser/xmlParser.cpp)
+add_subdirectory(CTML)
+target_include_directories(xmlParser PUBLIC xmlParser)
+target_link_libraries(${PROJECT_NAME} PRIVATE xmlParser CTML)
+target_include_directories(${PROJECT_NAME} PRIVATE .)
+target_compile_options(${PROJECT_NAME} PRIVATE -Wall -std=c++17)

hw_html/CTML

@@ -0,0 +1 @@
+Subproject commit 45cf94c34d31db1a30413a57617608a7c4bae343

hw_html/bit.cpp

@@ -0,0 +1,16 @@
+#include <bit.hpp>
+#include <iostream>
+
+bit::bit(const std::string &name, const std::string &access,
+    const std::string &pos) :
+    mcubase(name, access),
+    off(get(pos))
+{
+}
+
+struct bit::off bit::get(const std::string &pos)
+{
+    struct off off = {0};
+    std::cout << pos + "\n";
+    return off;
+}

hw_html/bit.hpp

@@ -0,0 +1,20 @@
+#ifndef BIT_HPP
+#define BIT_HPP
+
+#include <mcubase.hpp>
+#include <string>
+
+class bit : public mcubase
+{
+public:
+    bit(const std::string &name, const std::string &access,
+        const std::string &pos);
+    virtual ~bit() = default;
+
+protected:
+    const struct off {unsigned int upper, lower;} off;
+    static struct off get(const std::string &pos);
+    std::string comment;
+};
+
+#endif /* BIT_HPP */

hw_html/html.cpp

@@ -0,0 +1,49 @@
+#include <html.hpp>
+#include <mcubase.hpp>
+#include <ctml.hpp>
+#include <stdio.h>
+#include <string.h>
+#include <errno.h>
+
+html::html() :
+    f(nullptr)
+{
+}
+
+html::~html()
+{
+    if (f)
+        fclose(f);
+}
+
+html::error html::open(const char *const path)
+{
+    if (!path)
+        return html::INVALID_PATH;
+
+    f = fopen(path, "wb");
+
+    if (!f)
+    {
+        fprintf(stderr, "could not open %s for writing, reason: %s\n", path,
+            strerror(errno));
+        return html::OPEN_ERR;
+    }
+
+    CTML::Document doc;
+
+    doc.AppendNodeToHead(CTML::Node("TITLE", "A9 register map"));
+
+    if (fprintf(f, "%s\n", doc.ToString().c_str()) < 0)
+    {
+        fprintf(stderr, "HTML write error\n");
+        return html::WRITE_ERR;
+    }
+
+    return html::OK;
+}
+
+html::error html::add(const mcubase &reg)
+{
+    return html::OK;
+}

hw_html/html.hpp

@@ -0,0 +1,28 @@
+#ifndef HTML_HPP
+#define HTML_HPP
+
+#include <ctml.hpp>
+#include <mcubase.hpp>
+#include <stdio.h>
+
+class html
+{
+public:
+    enum error
+    {
+        OK,
+        INVALID_PATH,
+        OPEN_ERR,
+        WRITE_ERR
+    };
+
+    html();
+    virtual ~html();
+    error open(const char *path);
+    error add(const mcubase &reg);
+
+protected:
+    FILE *f;
+};
+
+#endif /* HTML_HPP */

hw_html/main.cpp

@@ -0,0 +1,93 @@
+#include "html.hpp"
+#include "mcureg.hpp"
+#include "bit.hpp"
+#include <xmlParser.h>
+#include <string>
+#include <stdio.h>
+#include <stdlib.h>
+
+int main(const int argc, const char *const argv[])
+{
+    if (argc != 3)
+    {
+        fprintf(stderr, "%s <8955_hard.xml-path> <out-path>\n", argv[0]);
+        return EXIT_FAILURE;
+    }
+
+    const char *const inpath = argv[1];
+    const char *const outpath = argv[2];
+
+    html html;
+
+    if (html.open(outpath))
+        return EXIT_FAILURE;
+
+    XMLResults res;
+    XMLNode root = XMLNode::parseFile(inpath, "bigarchive", &res);
+
+    if (res.error != eXMLErrorNone)
+    {
+        fprintf(stderr, "Open %s failed, line %d, column %d, reason: %s.\n",
+            inpath, res.nLine, res.nColumn, XMLNode::getError(res.error));
+        return EXIT_FAILURE;
+    }
+
+    int n = 0;
+
+    for (int i = 0; i < root.nChildNode("archive"); i++)
+    {
+        const XMLNode archive = root.getChildNode("archive", &n);
+        const char *const relative = archive.getAttribute("relative");
+
+        if (!relative)
+            continue;
+
+        int n = 0;
+
+        for (int i = 0; i < archive.nChildNode("module"); i++)
+        {
+            const XMLNode module = archive.getChildNode("module", &n);
+            const char *const name = module.getAttribute("name");
+
+            if (!name)
+                continue;
+
+            printf("module %s\n", name);
+            int n = 0;
+
+            for (int i = 0; i < module.nChildNode("reg"); i++)
+            {
+                const XMLNode reg = module.getChildNode("reg", &n);
+                const char *const name = reg.getAttribute("name");
+                const char *const access = reg.getAttribute("protect");
+
+                if (!name || !access)
+                    continue;
+
+                printf("\treg %s\n", name);
+                mcureg mcureg(name, access);
+
+                int n = 0;
+
+                for (int i = 0; i < reg.nChildNode("bits"); i++)
+                {
+                    const XMLNode bits = reg.getChildNode("bits", &n);
+                    const char *const name = bits.getAttribute("name");
+                    const char *const access = bits.getAttribute("access");
+                    const char *const pos = bits.getAttribute("pos");
+                    const char *const rst = bits.getAttribute("rst");
+
+                    printf("\t\tbit=%s, access=%s, pos=%s, rst=%s\n",
+                        name, access, pos, rst);
+
+                    if (!name || !access || !pos || !rst)
+                        continue;
+
+                    mcureg.add(bit(name, access, pos));
+                }
+            }
+        }
+    }
+
+    return EXIT_SUCCESS;
+}

hw_html/mcubase.cpp

@@ -0,0 +1,36 @@
+#include "mcubase.hpp"
+#include <string>
+#include <stdint.h>
+
+mcubase::mcubase(const std::string &name, const std::string &access) :
+    name(name),
+    acc(get(access)),
+    rst(0)
+{
+}
+
+mcubase::access mcubase::get(const std::string &access)
+{
+    if (access == "rsv")
+        return mcubase::RSV;
+    else if (access == "w1c")
+        return mcubase::W1C;
+    else if (access == "w1s")
+        return mcubase::W1S;
+    else if (access == "rc")
+        return mcubase::RC;
+    else if (access == "rs")
+        return mcubase::RS;
+    else if (access == "rw")
+        return mcubase::RW;
+    else if (access == "c")
+        return mcubase::C;
+    else if (access == "s")
+        return mcubase::S;
+    else if (access == "r")
+        return mcubase::R;
+    else if (access == "w")
+        return mcubase::W;
+
+    return mcubase::UNDEF;
+}

hw_html/mcubase.hpp

@@ -0,0 +1,23 @@
+#ifndef MCUBASE_HPP
+#define MCUBASE_HPP
+
+#include <stdint.h>
+#include <string>
+
+class mcubase
+{
+public:
+    enum access {UNDEF, C, R, S, W, RC, RS, RW, RSV, W1C, W1S};
+    mcubase(const std::string &name, const std::string &access);
+
+protected:
+    mcubase() = default;
+    const std::string &name;
+    const access acc;
+    uint32_t rst;
+
+private:
+    static access get(const std::string &access);
+};
+
+#endif /* MCUBASE_HPP */

hw_html/mcureg.cpp

@@ -0,0 +1,15 @@
+#include <mcureg.hpp>
+#include <mcubase.hpp>
+#include <bit.hpp>
+#include <list>
+#include <string>
+
+mcureg::mcureg(const std::string &name, const std::string &access) :
+    mcubase(name, access)
+{
+}
+
+void mcureg::add(const bit &bit)
+{
+    bits.push_back(bit);
+}

hw_html/mcureg.hpp

@@ -0,0 +1,20 @@
+#ifndef MCUREG_HPP
+#define MCUREG_HPP
+
+#include <mcubase.hpp>
+#include <bit.hpp>
+#include <list>
+#include <string>
+
+class mcureg : public mcubase
+{
+public:
+    mcureg(const std::string &name, const std::string &access);
+    virtual ~mcureg() = default;
+    void add(const bit &);
+
+protected:
+    std::list<bit> bits;
+};
+
+#endif /* MCUREG_HPP */

hw_html/xmlParser

@@ -0,0 +1 @@
+Subproject commit ad1e7bcf1f28980d83c442d1d10d47a5fd2c1851