diff --git a/README.md b/README.md index 7b8f785..3c86713 100644 --- a/README.md +++ b/README.md @@ -1,138 +1,171 @@ -# pySymProxy -pySymProxy is an implementation of a Microsoft Symbol Proxy server using Python. -Symbol proxy servers are used to organise, optimise, and cache debugging symbols from multiple other servers. - -![screenshot](static/screenshot.png "pySymProxy status page screenshot") - -See the following links for information on symbols and symbol servers - - [Debugging with Symbols](https://msdn.microsoft.com/en-us/library/windows/desktop/ee416588(v=vs.85).aspx) - - [Symbol Server and Symbol Stores](https://msdn.microsoft.com/en-us/library/windows/desktop/ms680693(v=vs.85).aspx) - - [SymProxy](https://msdn.microsoft.com/en-us/library/windows/hardware/ff558846(v=vs.85).aspx) - - [Symbols the Microsoft way](https://randomascii.wordpress.com/2013/03/09/symbols-the-microsoft-way/) - ---- -## Features -| Feature | Microsoft SymProxy | pySymProxy | Description | -| ------------- |:-------------:|:-----:|-----| -| Common configuration of symbol search paths | X| X| Configure clients once with the symbol server's details and then all future configuration changes only need to occur on one server. Entire studios can be updated from one place. | -| Global blacklist | X | X | Commonly unavailable Symbols can be denied immediately by the service and avoid unnecessary requests and delays to debuggers. | -| Per server blacklist | | X | Different servers house the artifacts of different builds. If a server will never have a symbol then black list it and don't send the request in the first place. Useful when symbols will exist on one of the servers you reference, so a global blacklist won't help. | -| Per server whitelist | | X | This might be easier to configure than a per server blacklist. | -| Statistics collection | | X | Logs and statistics about each symbol served are kept and made available on the web interface. These statistics may help you identify further configuration optimisation opportunities. | -| Configurable server retry timeouts | | X| There is no need to query servers over and over again for symbols. If they didn't exist there 5 minutes ago, is it likely they're there now? Configure the timeout that makes sense for each server. | -| Open source | | X | Need another feature? Jump in and implement it :)| ---- -## Requirements -| Requirement | Microsoft SymProxy | pySymProxy | -| ------------- |:-------------:|:-----:| -| Windows | X| X| -| IIS | X| | -| Python | | X| ---- -## Why -I was driven to implement this solution when I found myself waiting for symbols on my system to download and run all the time. The reason this had become intolerably slow was because one of the servers we might fetch symbols from exists on the other side of the world. I had configured this server to be last in my symbol search paths so other (faster) servers would get the chance to service the request first. This worked for most symbols, but there are plenty of modules that we simply don't have access to the symbols for. These would always hit the slow server (after hitting the others) and wait for that server to respond to each request (every symbol fetch requires 3 HTTP requests to the remote servers). Some of my colleagues had setup exclusion lists for their configurations, but it became aparrent that these configurations were non-trivial and completely different on each developer's machine. - -This sounded like the perfect reason to setup a SymProxy as described by the Microsoft documentation. After attempting configuration of such a proxy, it also became clear that configuring this proxy wasn't going to be easy, and it would still hit the slow path for many of the requests we would make. - ---- -## Configuration -### Installation -Make sure you are running windows and have Python 2.7 or 3+ installed. -You can clone this repository into a folder on the windows machine. -To run the server, you can run `run_server.bat` in the root folder of the repository. -This will run a server on the machine on port 8080. -`run_server.bat` can be configured as a service on the machine so it automatically starts when the machine reboots. A search online can help with this configuration. - -Due to Microsoft not explicitly allowing redistribution od the `dbghelp.dll` files anymore, I can't have them committed directly into the repository. But I can point you to the place to get them: [Debugging Tools for Windows](https://developer.microsoft.com/en-us/windows/hardware/windows-driver-kit). Simply copy/paste the x86 `dbghelp.dll` and `symsrv.dll` to the `dbghelp` directory of the repository. - -### Client configuration -Developers PC's should be configured to reference the pySymProxy service by setting an environment variable on their machine to the following: -`srv*C:\Symbols*http://pysymproxy.company.local:8080/symbols.` - -### Checking the status of the server -At any time navigate to the base address of the server in a web browser. -Such as: `http://pysymproxy.company.local:8080/` - -### Configuration files -When starting, the server will attempt to load a configuration file. -The following locations will be checked, and the first location that exists will be used as the configuration. -- `../config/pysymproxy.json` -- `./config/pysymproxy.json` -- `./config/default.pysymproxy.json` - -The repository has a `./config/default.pysymproxy.json`. It is recommended to copy this file to one of the other locations, then begin to configure the server for your needs. - -These files are standard JSON files and contain sections that define how the server is configured. - -#### Configuration - `identity` section -- `name` - (string) The title describing the server -- `host` - (string) The name of the host that the server is running on -- `administrator` - (string) Contact information for the server administrator (displayed on the status page), -- `default_sympath` - (string) Client configuration information offered on the status page. - -#### Configuration - `general` section -- `enableStatistics` - (boolean) Whether or not statistics should be enabled on the server - may affect performance -- `cacheLocation` - (string) A location on disk where symbols can be cached for serving to clients -- `blacklist` - (list of strings) A list of regular expression patterns. If a requested file matches one of these patterns it will be rejected immediately. - -#### Configuration - `servers` section -This section is expected to contain a list of server objects. -When attempting to find symbols, these servers will be searched in the order they are defined in this list. -Each object can have the following properies: -- `name` - (string) The name of this server -- `remote` - (string) a URL, file path, or network path where symbols are stored as a symbol server -- `cacheLocation` - (string) optional location on disk where symbols from this specific server can be cached for serving to clients -- `retryTimout` - (number) The number of seconds to wait after a failed symbol lookup before allowing another request for the same symbol to try again. -- `maxRequests` - (number) Maximum number of simultaneous requests that can be served by this server -- `blacklist` - (list of strings) A list of regular expression patterns. If a requested file matches one of these patterns it will be rejected immediately by this server. -- `whitelist` - (list of strings) A list of regular expression patterns. If a requested file does NOT match one of these patterns it will be rejected immediately by this server. - -#### Configuration - `logging` section -The config file can have a logging object defined. This defines a dict that is passed to Python's logging modules. This can be used to configure the server to generate log output to disk for later analysis. The default configuration outputs to rotating log files and to the console. See the Python logging documentation for specific details on this section - ---- -## Known issues -### "Failed to load symbol" -Sometimes when a symbol exists on a remote server, the remote requests aren't serviced quickly enough, and the client (debugger) requesting the symbol treats it as a failure. Often attempting to load the symbol again will hit the cache and immediately load the symbol the second time around. This might be solved in future using chunked encoding streams and generating 0 bytes to keep the connection alive for longer. - -### dbghelp.dll should not be accessed by multiple threads -Yeah, so we're abusing the dbghelp.dll a bit here. The documentation makes no guarantees that the dll will do what it should if it is accessed by multiple threads. This isn't really useful for a webservice - so in an attempt to limit the risk of something bad happening - I've loaded the dll into the process multiple times. Each load of the DLL will only be used by one thread at a time. This still may not guarantee things will work, and it may not even be required in this situation. EAFP is the 'pythonic way' right? - ---- -## FAQ -### I use local network shares for my symbol storage. Why would I use this? -You probably shouldn't. Local network shares are great and fast. -However, there are some useful features that may make it worth trying, such as statistics collection, and the fact that symbol server configuration is managed by the server rather than each developer's individual PC configuration. - -### Microsoft has an implementation of a Symbol Proxy (symproxy), why not use that? -Our initial attempts to configure a symproxy as per the documentation failed. Varying versions of IIS, mixed with configuration headaches and legacy installations made it unclear where the issue was. Whilst attempting to configure the server it also became clear that there were desirable configuration settings missing, and the complexity of the task that was being performed by the service wasn't very high. -So I thought I would give it a shot and write something that worked the way we wanted. - -### Why not just use linux? Why use windows? -When initially attempting an implementation, I tried to get a symproxy working using just python. This could then have been hosted on a linux server. I ran into trouble when attempting to forward requests to the Microsoft symbol server. This server doesn't accept requests from anything but their software. So the current implementation makes use of the dbghelp.dll and symsrv.dll to build requests for other servers (just as your debugger would). - -### How do I store symbols on this symbol server? -This server implementation just serves symbols. It can serve them from a local network share, so the recommended practice here is to "store" your built symbols on a local network share as you normally would. Then configure the server to serve the symbols stored at this location. - ---- -## Technologies used -- [Python](https://www.python.org/) -- [Falcon](https://falconframework.org/) -- [Waitress](http://docs.pylonsproject.org/projects/waitress/en/latest/) -- [Jinja2](http://jinja.pocoo.org/) -- [jQuery](https://jquery.com/) -- [w3-css](http://www.w3schools.com/w3css/) -- [Google Material Icons](https://material.io/icons/) -- [Debuging Tools for Windows](https://developer.microsoft.com/en-us/windows/hardware/windows-driver-kit) - ---- -## Disclaimer -Use at your own risk. -This is the first time I've written a webservice or used python for more than a small script. -If you think you could have done better - you probably could have :) -Please send me your pull requests! - ---- -## Support -Feel free to create issues if you see something that needs fixing or improvement. +# pySymProxy +pySymProxy is an implementation of a Microsoft Symbol Proxy server using Python. +Symbol proxy servers are used to organise, optimise, and cache debugging symbols from multiple other servers. + +![screenshot](static/screenshot.png "pySymProxy status page screenshot") + +See the following links for information on symbols and symbol servers + - [Debugging with Symbols](https://msdn.microsoft.com/en-us/library/windows/desktop/ee416588(v=vs.85).aspx) + - [Symbol Server and Symbol Stores](https://msdn.microsoft.com/en-us/library/windows/desktop/ms680693(v=vs.85).aspx) + - [SymProxy](https://msdn.microsoft.com/en-us/library/windows/hardware/ff558846(v=vs.85).aspx) + - [Symbols the Microsoft way](https://randomascii.wordpress.com/2013/03/09/symbols-the-microsoft-way/) + +--- +## Features +| Feature | Microsoft SymProxy | pySymProxy | Description | +| ------------- |:-------------:|:-----:|-----| +| Common configuration of symbol search paths | X| X| Configure clients once with the symbol server's details and then all future configuration changes only need to occur on one server. Entire studios can be updated from one place. | +| Global blacklist | X | X | Commonly unavailable Symbols can be denied immediately by the service and avoid unnecessary requests and delays to debuggers. | +| Per server blacklist | | X | Different servers house the artifacts of different builds. If a server will never have a symbol then black list it and don't send the request in the first place. Useful when symbols will exist on one of the servers you reference, so a global blacklist won't help. | +| Per server whitelist | | X | This might be easier to configure than a per server blacklist. | +| Statistics collection | | X | Logs and statistics about each symbol served are kept and made available on the web interface. These statistics may help you identify further configuration optimisation opportunities. | +| Configurable server retry timeouts | | X| There is no need to query servers over and over again for symbols. If they didn't exist there 5 minutes ago, is it likely they're there now? Configure the timeout that makes sense for each server. | +| Open source | | X | Need another feature? Jump in and implement it :)| +--- +## Requirements +| Requirement | Microsoft SymProxy | pySymProxy | +| ------------- |:-------------:|:-----:| +| Windows | X| X| +| IIS | X| | +| Python | | X| +--- +## Why +I was driven to implement this solution when I found myself waiting for symbols on my system to download and run all the time. The reason this had become intolerably slow was because one of the servers we might fetch symbols from exists on the other side of the world. I had configured this server to be last in my symbol search paths so other (faster) servers would get the chance to service the request first. This worked for most symbols, but there are plenty of modules that we simply don't have access to the symbols for. These would always hit the slow server (after hitting the others) and wait for that server to respond to each request (every symbol fetch requires 3 HTTP requests to the remote servers). Some of my colleagues had setup exclusion lists for their configurations, but it became aparrent that these configurations were non-trivial and completely different on each developer's machine. + +This sounded like the perfect reason to setup a SymProxy as described by the Microsoft documentation. After attempting configuration of such a proxy, it also became clear that configuring this proxy wasn't going to be easy, and it would still hit the slow path for many of the requests we would make. + +--- +## Configuration +Due to Microsoft not explicitly allowing redistribution od the `dbghelp.dll` files anymore, I can't have them committed directly into the repository. But I can point you to the place to get them: [Debugging Tools for Windows](https://developer.microsoft.com/en-us/windows/hardware/windows-driver-kit). Simply copy/paste the x86 `dbghelp.dll` and `symsrv.dll` to the `dbghelp` directory of the repository. + +### Installation +1. To run the server (port 5001), you can run `run_server.bat` in the root folder of the repository. + +### Run As Windows Service using WinSW +To run pySymProxy as a Windows Service, you can use [WinSW](https://github.com/winsw/winsw). WinSW is a simple wrapper that allows any executable to run as a Windows service. + +1. Download the latest `WinSW.exe` from the [WinSW releases page](https://github.com/winsw/winsw/releases). +2. Rename `WinSW.exe` to `pySymProxyService.exe` and place it in the root folder of the repository. +3. Create a configuration file named `pySymProxyService.xml` in the same folder. Example: + + ```xml + + pySymProxy + pySymProxy Service + Microsoft Symbol Proxy server implemented in Python + python + run_server.py + .\logs + + 10240 + 8 + + + ``` + +4. Open a command prompt as Administrator and run: + + ``` + pySymProxyService.exe install + ``` + +5. Start the service: + + ``` + pySymProxyService.exe start + ``` + +Refer to the [WinSW documentation](https://github.com/winsw/winsw/blob/main/doc/xml-config-file.md) for advanced configuration options. + + +### Client configuration +Developers PC's should be configured to reference the pySymProxy service by setting an environment variable on their machine to the following: +`srv*%LOCALAPPDATA%\Temp\SymbolCache*http://pysymproxy.company.local:5001/symbols.` + +### Checking the status of the server +At any time navigate to the base address of the server in a web browser. +Such as: `http://pysymproxy.company.local:5001/` + +### Configuration files +When starting, the server will attempt to load a configuration file. +The following locations will be checked, and the first location that exists will be used as the configuration. +- `../config/pysymproxy.json` +- `./config/pysymproxy.json` +- `./config/default.pysymproxy.json` + +The repository has a `./config/default.pysymproxy.json`. It is recommended to copy this file to one of the other locations, then begin to configure the server for your needs. + +These files are standard JSON files and contain sections that define how the server is configured. + +#### Configuration - `identity` section +- `name` - (string) The title describing the server +- `host` - (string) The name of the host that the server is running on +- `administrator` - (string) Contact information for the server administrator (displayed on the status page), +- `default_sympath` - (string) Client configuration information offered on the status page. + +#### Configuration - `general` section +- `enableStatistics` - (boolean) Whether or not statistics should be enabled on the server - may affect performance +- `cacheLocation` - (string) A location on disk where symbols can be cached for serving to clients +- `blacklist` - (list of strings) A list of regular expression patterns. If a requested file matches one of these patterns it will be rejected immediately. + +#### Configuration - `servers` section +This section is expected to contain a list of server objects. +When attempting to find symbols, these servers will be searched in the order they are defined in this list. +Each object can have the following properies: +- `name` - (string) The name of this server +- `remote` - (string) a URL, file path, or network path where symbols are stored as a symbol server +- `cacheLocation` - (string) optional location on disk where symbols from this specific server can be cached for serving to clients +- `retryTimout` - (number) The number of seconds to wait after a failed symbol lookup before allowing another request for the same symbol to try again. +- `maxRequests` - (number) Maximum number of simultaneous requests that can be served by this server +- `blacklist` - (list of strings) A list of regular expression patterns. If a requested file matches one of these patterns it will be rejected immediately by this server. +- `whitelist` - (list of strings) A list of regular expression patterns. If a requested file does NOT match one of these patterns it will be rejected immediately by this server. + +#### Configuration - `logging` section +The config file can have a logging object defined. This defines a dict that is passed to Python's logging modules. This can be used to configure the server to generate log output to disk for later analysis. The default configuration outputs to rotating log files and to the console. See the Python logging documentation for specific details on this section + +--- +## Known issues +### "Failed to load symbol" +Sometimes when a symbol exists on a remote server, the remote requests aren't serviced quickly enough, and the client (debugger) requesting the symbol treats it as a failure. Often attempting to load the symbol again will hit the cache and immediately load the symbol the second time around. This might be solved in future using chunked encoding streams and generating 0 bytes to keep the connection alive for longer. + +### dbghelp.dll should not be accessed by multiple threads +Yeah, so we're abusing the dbghelp.dll a bit here. The documentation makes no guarantees that the dll will do what it should if it is accessed by multiple threads. This isn't really useful for a webservice - so in an attempt to limit the risk of something bad happening - I've loaded the dll into the process multiple times. Each load of the DLL will only be used by one thread at a time. This still may not guarantee things will work, and it may not even be required in this situation. EAFP is the 'pythonic way' right? + +--- +## FAQ +### I use local network shares for my symbol storage. Why would I use this? +You probably shouldn't. Local network shares are great and fast. +However, there are some useful features that may make it worth trying, such as statistics collection, and the fact that symbol server configuration is managed by the server rather than each developer's individual PC configuration. + +### Microsoft has an implementation of a Symbol Proxy (symproxy), why not use that? +Our initial attempts to configure a symproxy as per the documentation failed. Varying versions of IIS, mixed with configuration headaches and legacy installations made it unclear where the issue was. Whilst attempting to configure the server it also became clear that there were desirable configuration settings missing, and the complexity of the task that was being performed by the service wasn't very high. +So I thought I would give it a shot and write something that worked the way we wanted. + +### Why not just use linux? Why use windows? +When initially attempting an implementation, I tried to get a symproxy working using just python. This could then have been hosted on a linux server. I ran into trouble when attempting to forward requests to the Microsoft symbol server. This server doesn't accept requests from anything but their software. So the current implementation makes use of the dbghelp.dll and symsrv.dll to build requests for other servers (just as your debugger would). + +### How do I store symbols on this symbol server? +This server implementation just serves symbols. It can serve them from a local network share, so the recommended practice here is to "store" your built symbols on a local network share as you normally would. Then configure the server to serve the symbols stored at this location. + +--- +## Technologies used +- [Python](https://www.python.org/) +- [Falcon](https://falconframework.org/) +- [Waitress](http://docs.pylonsproject.org/projects/waitress/en/latest/) +- [Jinja2](http://jinja.pocoo.org/) +- [jQuery](https://jquery.com/) +- [w3-css](http://www.w3schools.com/w3css/) +- [Google Material Icons](https://material.io/icons/) +- [Debuging Tools for Windows](https://developer.microsoft.com/en-us/windows/hardware/windows-driver-kit) + +--- +## Disclaimer +Use at your own risk. +This is the first time I've written a webservice or used python for more than a small script. +If you think you could have done better - you probably could have :) +Please send me your pull requests! + +--- +## Support +Feel free to create issues if you see something that needs fixing or improvement. I also welcome pull requests for useful additions and bugfixes. \ No newline at end of file diff --git a/api/dbghelp.py b/api/dbghelp.py index 7c2f868..5c129a2 100644 --- a/api/dbghelp.py +++ b/api/dbghelp.py @@ -1,6 +1,3 @@ -from __future__ import print_function -# Import print() function for Python 2.7 compatibility - import ctypes import ctypes.wintypes import os @@ -48,7 +45,7 @@ class CBA_EVENT_DATA(ctypes.Structure): ('object', ctypes.c_void_p)] data = ctypes.cast(callbackData, ctypes.POINTER(CBA_EVENT_DATA)) - message = data[0].desc.replace("\b", "").strip() + message = data[0].desc.decode('utf-8', errors='replace').replace("\b", "").strip() logger.info("dllEvent {}>({}) {}".format(self._uniqueProcessHandle, data[0].code, message)) return 1 elif actionCode == 0x07: @@ -82,7 +79,7 @@ def initialize(self): self.SymSetOptions(symoptions) # Initialize the symbol system - success = self.SymInitialize(self._uniqueProcessHandle, ctypes.c_char_p(self._sympath), ctypes.c_bool(False)) + success = self.SymInitialize(self._uniqueProcessHandle, ctypes.c_char_p(self._sympath.encode('utf-8')), ctypes.c_bool(False)) if (success == False): raise ctypes.WinError() @@ -179,7 +176,12 @@ def findFile_Binary(self, name, identifier): fileLocation = ctypes.create_string_buffer(b'\000' * 1024) flags = self.SSRVOPT_DWORD - result = self.SymFindFileInPath(self._uniqueProcessHandle, self._sympath, name, id1, id2, 0, flags, fileLocation, None, None) + result = self.SymFindFileInPath( + self._uniqueProcessHandle, + self._sympath.encode('utf-8'), + name.encode('utf-8'), + id1, id2, 0, flags, + fileLocation, None, None) if (not result): raise ctypes.WinError() @@ -194,8 +196,12 @@ def findFile_Pdb(self, name, identifier): fileLocation = ctypes.create_string_buffer(b'\000' * 1024) flags = self.SSRVOPT_GUIDPTR - result = self.SymFindFileInPath_pdb(self._uniqueProcessHandle, self._sympath, name, ctypes.byref(id1), id2, 0, - flags, fileLocation, None, None) + result = self.SymFindFileInPath_pdb( + self._uniqueProcessHandle, + self._sympath.encode('utf-8'), + name.encode('utf-8'), + ctypes.byref(id1), id2, 0, + flags, fileLocation, None, None) # if the search reports unsuccessful, it is possible it still # succeeded. This appears common with long distance servers with high latency. diff --git a/api/main.py b/api/main.py index 54537f1..cc1d688 100644 --- a/api/main.py +++ b/api/main.py @@ -15,7 +15,7 @@ testrouteHandler = testhandler.TestHandler() defaultroutehandler = mainhandler.MainHandler(configuration, symbolroutehandler.getStats()) -api = falcon.API() +api = falcon.App() api.add_route('/{file}', defaultroutehandler) api.add_route('/symbols/{file}/{identifier}/{rawfile}', symbolroutehandler) api.add_route('/test/{file1}/{file2}/{file3}', testrouteHandler) \ No newline at end of file diff --git a/api/mainhandler.py b/api/mainhandler.py index c3a6412..3cc61b6 100644 --- a/api/mainhandler.py +++ b/api/mainhandler.py @@ -53,13 +53,13 @@ def on_get(self, req, resp, file): elif (file.endswith(".log")): return self.on_get_logfile(req, resp, file) except Exception as e: - resp.body = "error: " + str(e) + resp.text = "error: " + str(e) def on_get_index(self, req, resp): diskUsage = 0 # sum([getFolderSize(server.get("cacheLocation", None)) for server in self._config.servers()]) diskUsage += getFolderSize(self._config.cacheLocation()) self._template = env.get_template('main.html.jinja') - resp.body = self._template.render( + resp.text = self._template.render( serverName=self._config.name(), admin=self._config.administrator(), clientConfig=self._config.sympath(), @@ -69,13 +69,14 @@ def on_get_index(self, req, resp): diskUsage=diskUsage, logfiles=self._config.logfiles() ) - resp.content_type = "html" + + resp.content_type = "text/html" def on_get_config(self, req, resp): configLocation = self._config.configFile() resp.stream = open(configLocation, 'rb') resp.stream_len = os.path.getsize(configLocation) - resp.content_type = "json" + resp.content_type = "application/json" def on_get_statistics(self, req, resp): # Build a dictionary of information to send @@ -85,16 +86,16 @@ def on_get_statistics(self, req, resp): stats.diskUsage += getFolderSize(self._config.cacheLocation()) stats.numAcceptedRequests = stats.numRequests.value - stats.numExcluded.value - resp.data = JsonEncoder().encode(stats) - resp.content_type = "json" + resp.text = JsonEncoder().encode(stats) + resp.content_type = "application/json" def on_get_symbols(self, req, resp): # Build a dictionary of information to send # Serialise it and send symbols = self._statistics.getSymbols() - resp.data = JsonEncoder().encode(symbols) - resp.content_type = "json" + resp.text = JsonEncoder().encode(symbols) + resp.content_type = "application/json" def on_get_logfile(self, req, resp, file): # Get the list of log files @@ -104,4 +105,4 @@ def on_get_logfile(self, req, resp, file): logLocation = logfiles[logIndex - 1] resp.stream = open(logLocation, 'rb') resp.stream_len = os.path.getsize(logLocation) - resp.content_type = "text" + resp.content_type = "text/plain" diff --git a/api/objectpool.py b/api/objectpool.py index 39abad5..07bbbcb 100644 --- a/api/objectpool.py +++ b/api/objectpool.py @@ -1,8 +1,4 @@ -# Python 3+ has module "queue", while 2.7 has module "Queue" -try: - import queue -except ImportError: - import Queue as queue +import queue from contextlib import contextmanager import threading diff --git a/api/requeststatistics.py b/api/requeststatistics.py index 48ade9e..3968157 100644 --- a/api/requeststatistics.py +++ b/api/requeststatistics.py @@ -36,17 +36,19 @@ def __init__(self, file): self.serverHits = {} def recordServerHit(self, server): - counter = self.serverHits.get(server.identifer(), None) + sid = server.identifier() + counter = self.serverHits.get(sid, None) if counter is None: counter = AtomicCounter() - self.serverHits[server.identifer()] = counter + self.serverHits[sid] = counter counter.increment() def recordServerMiss(self, server): - counter = self.serverMisses.get(server.identifer(), None) + sid = server.identifier() + counter = self.serverMisses.get(sid, None) if counter is None: counter = AtomicCounter() - self.serverMisses[server.identifer()] = counter + self.serverMisses[sid] = counter counter.increment() def encodeJSON(self): diff --git a/api/symbolhandler.py b/api/symbolhandler.py index 3464391..8b94afa 100644 --- a/api/symbolhandler.py +++ b/api/symbolhandler.py @@ -80,7 +80,8 @@ def on_get(self, req, resp, file, identifier, rawfile): except Exception as e: logging.error("{}".format(str(e))) - resp.body = "404 could not find requested file.\nError: " + str(e) + resp.text = "404 could not find requested file.\nError: " + str(e) + resp.content_type = "text/plain" resp.status = falcon.HTTP_404 self._statistics.endRequest(statRecord, file, identifier, symbolLocation, cacheHit, excluded, valid, diff --git a/api/symbolserver.py b/api/symbolserver.py index f045354..096fb43 100644 --- a/api/symbolserver.py +++ b/api/symbolserver.py @@ -79,5 +79,9 @@ def findFile(self, file, identifier): self._previousResults[recordId] = newRecord return newRecord.location, False, True - def identifer(self): + # Backwards-compatible accessor for the server identifier + def identifier(self): return self._identifier + + # Temporary shim to support older code paths referencing the misspelled name + identifer = identifier diff --git a/api/testhandler.py b/api/testhandler.py index fc6e427..0b1349a 100644 --- a/api/testhandler.py +++ b/api/testhandler.py @@ -16,4 +16,5 @@ def on_get(self, req, resp, file1, file2, file3): resp.content_type = "application/octet-stream" except Exception as e: - resp.body = "error: " + str(e) + resp.text = "error: " + str(e) + resp.content_type = "text/plain" diff --git a/config/default.pysymproxy.json b/config/default.pysymproxy.json index 0c64a61..eaf1d83 100644 --- a/config/default.pysymproxy.json +++ b/config/default.pysymproxy.json @@ -3,7 +3,7 @@ "name": "PySymProxy - Default configuration", "host": "localhost", "administrator": "!enter your contact details!", - "default_sympath": "srv*c:\\symbols*http://localhost:8080/symbols" + "default_sympath": "srv*%LOCALAPPDATA%\\Temp\\SymbolCache*http://localhost:5001/symbols" }, "general": { "enableStatistics": true, @@ -20,7 +20,7 @@ { "name": "Microsoft symbol server", "identifier": "mss", - "remote": "http://msdl.microsoft.com/download/symbols", + "remote": "https://msdl.microsoft.com/download/symbols", "cacheLocation": "\\\\build-server\\mssymbols", "retryTimout": 600, "maxRequests": 10 diff --git a/dbghelp/symsrv.yes b/dbghelp/symsrv.yes index 56a6051..0519ecb 100644 --- a/dbghelp/symsrv.yes +++ b/dbghelp/symsrv.yes @@ -1 +1 @@ -1 \ No newline at end of file + \ No newline at end of file diff --git a/requirements.txt b/requirements.txt index d42d548..c97d9a0 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,4 +1,5 @@ -falcon -requests -waitress -jinja2 \ No newline at end of file +falcon +requests +waitress +jinja2 +markupsafe \ No newline at end of file diff --git a/server.py b/server.py index e4f820f..903b844 100644 --- a/server.py +++ b/server.py @@ -1,14 +1,11 @@ -from __future__ import print_function -# Import print() function for Python 2.7 compatibility - from api.main import api from waitress import serve import logging logger = logging.getLogger(__name__) logger.info("Starting SymProxy Server") -print("Test link: http://localhost:8080/symbols/wntdll.pdb/F999943DF7FB4B8EB6D99F2B047BC3101/wntdll.pdb") -serve(api, host='0.0.0.0', port=8080) +print("Test link: http://localhost:5001/symbols/wntdll.pdb/F999943DF7FB4B8EB6D99F2B047BC3101/wntdll.pdb") +serve(api, host='0.0.0.0', port=5001) #TODO: # - Add lazy evaluation of storage space