Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
90 commits
Select commit Hold shift + click to select a range
0574d5d
Add default.nix
acairncross Jul 11, 2023
ef50554
Bump dense Nix to most recent
acairncross Jul 12, 2023
0ff09fd
Add a way to not use cabal.project
acairncross Jul 12, 2023
5a4783c
Add instructions for cabal2nix
acairncross Jul 12, 2023
3fd17d7
Clean up shell.nix
acairncross Jul 12, 2023
0beb872
Add build instructions
acairncross Jul 12, 2023
bb795bf
Remove unneeded things from nixpkgs overlay
acairncross Jul 12, 2023
fe4f168
copied across updated set up
Rowan-Mather Jul 12, 2023
c17388e
first build
Rowan-Mather Jul 12, 2023
36da8c3
implemented dtypes
Rowan-Mather Jul 12, 2023
c772873
pointwise generic in progress
Rowan-Mather Jul 12, 2023
cf59e8f
testing serialisation
Rowan-Mather Jul 13, 2023
b5b2f63
still trying to get serialisation to work
Rowan-Mather Jul 13, 2023
e3c45f6
123! serialisation!
Rowan-Mather Jul 13, 2023
34835d8
basile fixed pointwise!
Rowan-Mather Jul 13, 2023
7881d3e
eq, ord & tidying
Rowan-Mather Jul 13, 2023
394f26d
just before I break everything by adding shapes
Rowan-Mather Jul 14, 2023
167f93e
added shapes & it compiles, fixed ord, ord => dtyp
Rowan-Mather Jul 14, 2023
23e86e9
working on reshaping
Rowan-Mather Jul 14, 2023
88923ff
resizing and padding
Rowan-Mather Jul 17, 2023
d404264
many serialisation updates
Rowan-Mather Jul 17, 2023
aa9f7ea
serialised numpy to haskell
Rowan-Mather Jul 18, 2023
ebea448
tidying
Rowan-Mather Jul 18, 2023
57358bf
added git ignore
Rowan-Mather Jul 18, 2023
402b4c0
tidying
Rowan-Mather Jul 18, 2023
4b040ce
changed package name & added matrix
Rowan-Mather Jul 19, 2023
204957c
more creation & indexing that works
Rowan-Mather Jul 20, 2023
e8087d8
pretty printing
Rowan-Mather Jul 20, 2023
278700a
Much numskull documentation
Rowan-Mather Jul 20, 2023
1d4fd01
working on matrix multiplication
Rowan-Mather Jul 21, 2023
fa928ba
matrix multiplication
Rowan-Mather Jul 24, 2023
3b8d005
matrix multiplication
Rowan-Mather Jul 24, 2023
ff951ff
working on conversions
Rowan-Mather Jul 24, 2023
443a938
Dtype conversions done
Rowan-Mather Jul 25, 2023
face528
serialisation generalisation progess
Rowan-Mather Jul 25, 2023
43ceb84
Another serialisation update
Rowan-Mather Jul 25, 2023
1424d1f
Transposition and more serialisation
Rowan-Mather Jul 26, 2023
e33f8a9
working on determinants
Rowan-Mather Jul 26, 2023
83d67a3
2d determinant done compiles but untested
Rowan-Mather Jul 27, 2023
37c34ad
determinants works
Rowan-Mather Jul 28, 2023
e72d97f
tidying
Rowan-Mather Jul 28, 2023
dcc7e0d
slicing
Rowan-Mather Jul 28, 2023
cf751b4
dtype tidying
Rowan-Mather Jul 28, 2023
7134563
tidying
Rowan-Mather Jul 31, 2023
3aad0e7
trying to make nix work again
Rowan-Mather Jul 31, 2023
41ce678
broadcasting
Rowan-Mather Jul 31, 2023
d0a19b7
gathering & concatenating
Rowan-Mather Aug 1, 2023
af33a5e
gathering & concatenating
Rowan-Mather Aug 1, 2023
3ee6c7c
tidying & max/min
Rowan-Mather Aug 1, 2023
93b4a73
add to cabal
Rowan-Mather Aug 2, 2023
38755e0
export tidying
Rowan-Mather Aug 2, 2023
b8ee0a6
add vector constructor
Rowan-Mather Aug 2, 2023
bda8f45
exporting more stuff
Rowan-Mather Aug 2, 2023
4ac4899
more exports
Rowan-Mather Aug 2, 2023
396d757
tidying
Rowan-Mather Aug 3, 2023
e9dc3b3
ndtype string
Rowan-Mather Aug 3, 2023
761f6e2
working on gemm
Rowan-Mather Aug 3, 2023
3db5841
gemm compiles
Rowan-Mather Aug 3, 2023
7b1ac6b
fixed transpose
Rowan-Mather Aug 3, 2023
8dc0151
trying dot with gemm
Rowan-Mather Aug 4, 2023
e1b708b
debug gemm info
Rowan-Mather Aug 4, 2023
2ad2c15
more gemm debug
Rowan-Mather Aug 4, 2023
2f1b286
stumped
Rowan-Mather Aug 4, 2023
bed9229
urgh its concat thats causing issues :((
Rowan-Mather Aug 4, 2023
ec77ae6
i think its finally ok
Rowan-Mather Aug 4, 2023
fd22cf8
generalised matmul
Rowan-Mather Aug 8, 2023
a6f2bd0
summation
Rowan-Mather Aug 8, 2023
382f1d3
much tidying
Rowan-Mather Aug 9, 2023
8ca2ed1
hlinted everything
Rowan-Mather Aug 10, 2023
3f2f13e
adding exception to cabal
Rowan-Mather Aug 10, 2023
58de203
small fixes
Rowan-Mather Aug 24, 2023
d72ddfd
more efficient slice
Rowan-Mather Aug 24, 2023
e560f6a
new parser work for pretty slicing
Rowan-Mather Aug 16, 2023
d19b866
updated readme
Rowan-Mather Aug 24, 2023
8d7fe11
slicing merged and corrected
Rowan-Mather Aug 24, 2023
0d9d079
tidying
Rowan-Mather Aug 24, 2023
8846b50
Add files via upload
Rowan-Mather Aug 24, 2023
88033f0
Delete .vscode directory
Rowan-Mather Aug 24, 2023
60964f0
Create temp
Rowan-Mather Aug 24, 2023
a67db1d
Add files via upload
Rowan-Mather Aug 24, 2023
e7be68e
Delete temp
Rowan-Mather Aug 24, 2023
95151da
Update README.md
Rowan-Mather Aug 24, 2023
4b9d29a
s
Rowan-Mather Aug 25, 2023
056855a
Remove TemplateHaskell extension
acairncross Sep 26, 2023
ad3099a
Add NFData instance
acairncross Sep 26, 2023
9d43568
Add tests to cabal file
acairncross Oct 3, 2023
9895df0
Add upper bound on base dep
acairncross Oct 3, 2023
fb9711e
Remove invalid cabal
acairncross Oct 3, 2023
7db8be0
Re-run cabal2nix
acairncross Oct 4, 2023
7b83bb6
Numskull 2.0
Rowan-Mather Aug 10, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
dist-newstyle/
result/
.vscode/
30 changes: 30 additions & 0 deletions .hlint.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
#####################################################################
## HINTS

- functions:
- {name: Test.Hspec.focus, within: []} # focus should only be used for debugging
- {name: Prelude.undefined, within: []} # Prelude.undefined should only be used temporarily
- {name: Clash.XException.undefined, within: []} # Clash undefined should only be used temporarily (use deepErrorX instead)

- error: {lhs: fromString (show x), rhs: Data.String.Extra.show' x}
- error: {lhs: Data.Text.pack (show x), rhs: Data.String.Extra.show' x}
- error: {lhs: fromIntegral (Clash.Promoted.Nat.snatToInteger x), rhs: Clash.Promoted.Nat.snatToNum x}
- error: {lhs: Clash.Sized.Internal.BitVector.split# (ClaSH.Class.BitPack.pack x), rhs: Clash.Prelude.BitIndex.split}
- error: {lhs: Clash.Signal.mux p (fmap Just x) (pure Nothing), rhs: Clash.Signal.Extra.boolToMaybe p x}
- error: {lhs: Clash.Signal.mux p (Just <$> x) (pure Nothing), rhs: Clash.Signal.Extra.boolToMaybe p x}
- error: {lhs: Clash.Prelude.moore x id, rhs: Clash.Prelude.Moore.medvedev x}
- error: {lhs: Clash.Prelude.medvedev f x (pure ()), rhs: Clash.Source.source' (flip f ()) x}

# We tend to use pure over return
- error: {lhs: return, rhs: pure}
- error: {lhs: ceiling (logBase 2 (fromIntegral x)), rhs: Numeric.Natural.Extra.fromNatural (Numeric.Log2.clog2 (fromIntegral x))}
- error: {lhs: floor (logBase 2 (fromIntegral x)), rhs: Numeric.Natural.Extra.fromNatural (Numeric.Log2.flog2 (fromIntegral x))}
- error: {lhs: ceiling (logBase 4 (fromIntegral x)), rhs: Numeric.Natural.Extra.fromNatural (Numeric.Log2.clog4 (fromIntegral x))}
- error: {lhs: floor (logBase 4 (fromIntegral x)), rhs: Numeric.Natural.Extra.fromNatural (Numeric.Log2.flog4 (fromIntegral x))}

# We all know when it's appropriate to use [Char]
- ignore: {name: Use String}
- ignore: {name: Use head}
- ignore: {name: Reduce duplication}
- ignore: {name: Use tuple-section}
- ignore: {name: Use <$>}
171 changes: 171 additions & 0 deletions .stylish-haskell.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,171 @@
# stylish-haskell configuration file
# ==================================

# The stylish-haskell tool is mainly configured by specifying steps. These steps
# are a list, so they have an order, and one specific step may appear more than
# once (if needed). Each file is processed by these steps in the given order.
steps:
# Convert some ASCII sequences to their Unicode equivalents. This is disabled
# by default.
# - unicode_syntax:
# # In order to make this work, we also need to insert the UnicodeSyntax
# # language pragma. If this flag is set to true, we insert it when it's
# # not already present. You may want to disable it if you configure
# # language extensions using some other method than pragmas. Default:
# # true.
# add_language_pragma: true

# Align the right hand side of some elements. This is quite conservative
# and only applies to statements where each element occupies a single
# line.
- simple_align:
cases: false
top_level_patterns: false
records: true

# Import cleanup
- imports:
# There are different ways we can align names and lists.
#
# - global: Align the import names and import list throughout the entire
# file.
#
# - file: Like global, but don't add padding when there are no qualified
# imports in the file.
#
# - group: Only align the imports per group (a group is formed by adjacent
# import lines).
#
# - none: Do not perform any alignment.
#
# Default: global.
align: group

# Folowing options affect only import list alignment.
#
# List align has following options:
#
# - after_alias: Import list is aligned with end of import including
# 'as' and 'hiding' keywords.
#
# > import qualified Data.List as List (concat, foldl, foldr, head,
# > init, last, length)
#
# - with_alias: Import list is aligned with start of alias or hiding.
#
# > import qualified Data.List as List (concat, foldl, foldr, head,
# > init, last, length)
#
# - new_line: Import list starts always on new line.
#
# > import qualified Data.List as List
# > (concat, foldl, foldr, head, init, last, length)
#
# Default: after_alias
list_align: after_alias

# Long list align style takes effect when import is too long. This is
# determined by 'columns' setting.
#
# - inline: This option will put as much specs on same line as possible.
#
# - new_line: Import list will start on new line.
#
# - new_line_multiline: Import list will start on new line when it's
# short enough to fit to single line. Otherwise it'll be multiline.
#
# - multiline: One line per import list entry.
# Type with contructor list acts like single import.
#
# > import qualified Data.Map as M
# > ( empty
# > , singleton
# > , ...
# > , delete
# > )
#
# Default: inline
long_list_align: new_line

# List padding determines indentation of import list on lines after import.
# This option affects 'list_align' and 'long_list_align'.
list_padding: 2

# Separate lists option affects formating of import list for type
# or class. The only difference is single space between type and list
# of constructors, selectors and class functions.
#
# - true: There is single space between Foldable type and list of it's
# functions.
#
# > import Data.Foldable (Foldable (fold, foldl, foldMap))
#
# - false: There is no space between Foldable type and list of it's
# functions.
#
# > import Data.Foldable (Foldable(fold, foldl, foldMap))
#
# Default: true
separate_lists: true

# Language pragmas
- language_pragmas:
# We can generate different styles of language pragma lists.
#
# - vertical: Vertical-spaced language pragmas, one per line.
#
# - compact: A more compact style.
#
# - compact_line: Similar to compact, but wrap each line with
# `{-#LANGUAGE #-}'.
#
# Default: vertical.
style: vertical

# Align affects alignment of closing pragma brackets.
#
# - true: Brackets are aligned in same collumn.
#
# - false: Brackets are not aligned together. There is only one space
# between actual import and closing bracket.
#
# Default: true
align: false

# stylish-haskell can detect redundancy of some language pragmas. If this
# is set to true, it will remove those redundant pragmas. Default: true.
remove_redundant: true

# Replace tabs by spaces. This is disabled by default.
# - tabs:
# # Number of spaces to use for each tab. Default: 8, as specified by the
# # Haskell report.
# spaces: 8

# Remove trailing whitespace
- trailing_whitespace: {}

# A common setting is the number of columns (parts of) code will be wrapped
# to. Different steps take this into account. Default: 80.
columns: 100

# By default, line endings are converted according to the OS. You can override
# preferred format here.
#
# - native: Native newline format. CRLF on Windows, LF on other OSes.
#
# - lf: Convert to LF ("\n").
#
# - crlf: Convert to CRLF ("\r\n").
#
# Default: native.
newline: lf

# Sometimes, language extensions are specified in a cabal file or from the
# command line instead of using language pragmas in the file. stylish-haskell
# needs to be aware of these, so it can parse the file correctly.
#
# No language extensions are enabled by default.
language_extensions:
- MultiParamTypeClasses
- FlexibleContexts
12 changes: 0 additions & 12 deletions DenseTest.hs

This file was deleted.

55 changes: 54 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1 +1,54 @@
# rowan-ndarray
# rowan-numskull

## Using Numskull

This is a Summer Internship project from 2023. Numskull is a NumPy-like library for Haskell, featuring NdArrays which can be created and manipulated to store many different types (of the DType class).

Numskull was designed for purposes of integration into an [Onnx](https://onnx.ai/) backend, but it can be used anywhere you need to operate on arrays of unspecified type and shape.

For more information, have a look at my talk: [slides](demo/presentation-slides.pdf).

To run the demo you need
1) jupyter
2) iHaskell (https://github.com/IHaskell/IHaskell) to put Numskull
code into a jupyter notebook.
3) nix-shell
4) cd demo/notebook/
5) ./start.sh

Note that the work in main is Numskull 1.0.
Numskull 2.0 can be found in the so-called branch! The second version is less well tested and complete, but should be more efficient since it makes use of strides. I didn't have time to integrate that into the Onnx backend, but it shouldn't be at all difficult to do so. There is an open pull request so it's easy to find.

## Development

### Using Cabal

This builds like any Cabal project with `cabal build`, `cabal repl`, etc.

### Using Nix (and Cabal)

There is a `default.nix` so the project can be built with `nix-build`, and a
`shell.nix` for a development shell with `nix-shell`.

#### Niv

Dependencies are maintained using `niv` in `nix/sources.json`.
Source repositories specified in `cabal.project` should be kept up to date with
`nix/sources.json`.

#### cabal2nix

`numskull.nix` should be updated with

```sh
$ cabal2nix . > numskull.nix
```

whenever the Cabal file is updated with e.g. new dependencies.

#### Nix shell

Within a Nix shell, ideally you build the project with
`cabal build --project-file=cabal-nix.project` to avoid fetching and building
dependencies specifid in `cabal.project` which are only intended for the
non-Nix build.
115 changes: 0 additions & 115 deletions TypeableTest.hs

This file was deleted.

Loading