Building a CSV viewer using TableView

Hello!

So I set myself this challenge as an introduction to HaxeUI (and a bit more of an introduction to Haxe). I’d basically like to have a cross platform CSV viewer which ideally works for reasonably large CSV files, and works something like the table UI for sqlitebrowser:

HaxeUI is quite impressive, with a lot of examples and an online builder which helped. It started off well, but then I ran straight into the problem in which I discovered that TableViews like to have their columns defined up-front. Constructing them on the fly to match the columns in a CSV file requires jumping through some hoops, as described here:

That post helped a lot: I’ve got dynamic columns working.

(At this point, I’ll add an aside: here’s an extra pointer to anyone else who tries this, which took me a while poring over the HaxeUI source code to work. Don’t use StringMaps to define the DataSource rows. Basically they won’t at all unless the HaxeUI code is refactored. In order to get dynamic DataSources to work, you have to use reflection to build anonymous structures with the right fields, matching the id properties set on the columns (see example below). This is a bit weird - it seems to imply that dynamic TableView columns is not a normal use-case at all! When I work out where to make feature requests, this may be the first and most obvious one…)

But there’s more.

I’m still having problems customising the column headers in a way which works on the fly. I want to have column headers have labels with filters below them, as per sqlitebrowser. I’ve dug through the source code some more, but it’s starting to take silly amounts of time to make progress. Therefore I think it’s time to ask for help…

(This was also compounded a bit with problems to get Haxe building on a relatively fast platform with sys, like Hashlink, on current LTS Ubuntu Linux. That’s another story though.)

The code as it is now (broken!) is here GitHub - wu-lee/haxe-csv at haxeui-community-wip

I’ll paste a specific part relevant to my question, which is the load() method from src/MainView.hx.

My question: I haven’t worked out what I need to do to make my custom column headers expand to a sensible size - they’re appearing as 12 pixels width, but I want them to expand to the size of the header text which is set, at least. (Ideally to the size of the widest data field, up to some limit.)

How do I do that?

I’d also like to have the columns resizable by dragging the edges - is this something TableView can do? I’ve not seen any demos which seem to show that.

Likewise, I’d ideally like to have long fields truncated (which I seem to have managed) but then have tooltips which show the full content (which I haven’t - again, it’s not clear how to do this dynamically.)

Thanks in advance for any help anyone can offer.

MainView.hx

// ...

@:build(haxe.ui.ComponentBuilder.build("assets/main-view.xml"))
class MainView extends VBox {
	public function new() {
		super();
	}

// ...

	// Defines how we generate a colum ID from its index
	private function colId(ix:Int) {
		return "c" + ix;
	}

	private function loadString(data:String) {
		trace("downloading "+data.substr(0,100));

		var reader = new Reader();
		reader.open(data);
		load(reader);
	}

	private function loadFile(file:SelectedFileInfo) {
		trace("load " + file.name);
		this.topComponent.screen.title = file.name;

		#if sys
		var stream = sys.io.File.read(file.fullPath, false);
		var reader = new Reader();
		reader.open(stream);
		load(reader);
		#end
	}

	private function load(reader:Reader) {
		// Populate a new datasource and headers array from the CSV first
		var headers:Array<String> = null;
		var ds = new ArrayDataSource<Dynamic>();
		for (record in reader) {
			if (headers == null) {
				headers = record;
				continue;
			}
			// We're forced by TableView's implementation to use anonymous objects and reflection
			// rather than the probably preferable StringMap
			var item:Dynamic = {};
			for (ix in 0...record.length) {
				var header = headers[ix];
				Reflect.setField(item, colId(ix), record[ix]);
			}
			ds.add(item);
		}

		// Now add the headers we found as table columns
		tv.clearContents(true);
		for (ix in 0...headers.length) {
			var header = headers[ix];
			var col = tv.addColumn(header);
			trace("size0 "+col.percentWidth+" / "+col.width);
			//col.autoSize();
			col.removeAllComponents();
			col.sortable = true;
			col.id = colId(ix);
			var content = ComponentBuilder.fromFile("assets/column.xml");
			var label = content.findComponent(Label, true);
			if (label != null)
				label.text = header;

			// None of the following seems to work: the columns all get added
			// with an unreadably narrow width.
			trace("size1 "+col.percentWidth+" / "+col.width);
			content.autoWidth = true;
			//content.autoSize();
			col.addComponent(content);
			trace("size2 "+col.percentWidth+" / "+col.width+" / "+content.width);
			//col.autoSize();
			trace("size3 "+col.percentWidth+" / "+col.width);
			col.autoWidth = true; 
		}


		// Now reset the renderers for the columns.
		// We have to do this or the renderers won't be appropriate.
		// See https://community.haxeui.org/t/dynamic-tableview/299/6
		tv.itemRenderer.removeAllComponents();
		for (ix in 0...headers.length) {
			var header = headers[ix];

			var ir = new ItemRenderer();
			var label = new Label();
			label.percentWidth = 100;
			label.verticalAlign = "center";
			label.id = colId(ix);
			label.autoHeight = false;
			label.clip = true;
			label.wordWrap = false;
			label.onChange = (e) -> {
				e.target.tooltip = e.target.value; // Doesn't work - would like to show the value
			};
			ir.addComponent(label);
			tv.itemRenderer.addComponent(ir);
		}
		tv.dataSource = ds;
	}

}

main-view.xml

<vbox width="100%" height="100%">
  <style>
     #vb1  {
    }
  </style>
  <menubar width="100%">
    <menu text="File">
      <menuitem id="menuItemLoad" text="Load" shortcutText="Ctrl+L" />
      <menuitem id="menuItemDownload" text="Download" shortcutText="Ctrl+D" />
      <menuitem id="menuItemSave" text="Save" disabled="true" shortcutText="Ctrl+S" />
      <menuitem id="menuItemSaveAs" text="Save As" disabled="true" shortcutText="Ctrl+Shift+S" />
    </menu>
  </menubar>
  <vbox id="vb1" width="100%" height="100%">
    <tableview id="tv" width="100%"  height="100%" contentWidth="100%">
      <header></header>
      <data></data>
    </tableview>
  </vbox>
</vbox>

column.xml

<itemrenderer width="100%">
    <column width="100%">
        <vbox width="100%">
            <label text="label" width="100%"/>
            <textfield placeholder="Filter" width="100%" />
        </vbox>
    </column>
</itemrenderer>

Howdy (and welcome!) :wave:

So, i thought i would try and create (and reuse) a dynamic table, just to see where i got - and for sure, there were issues - issues that ive fixed now: http://haxeui.org/builder/?3656bd32

Note that the builder is using latest haxeui-core (and backend) so you’ll need that locally for this to work. If you (or I) had used “addColumn” rather than building the header from scratch, everything would have been fine, but the fact even I didnt remember that was the case wasnt ideal - so now you can use either method. There is a slight “flash” as the data changes, which ill look at at some point, but it might be unavoidable - not sure yet.

When it comes to native, the reuse doesnt seem to work properly, well, it changes the column headers, but somehow loses the data, another thing ill look at (though creating a brand new dynamic table seems fine on all backends).

Its probably worth mentioning, but things like having filters (text inputs) in your headers is not going to work for native. Its not something that native tableviews will allow, so is completely out of my hands.

Let me know if that helps somewhat and if you run into any other issues.

Cheers!
Ian

Yes, thanks for the speedy reply, this helps, and I’ll have another go.

I did think I was using addColumn() however… yes, second line of the headers loop in load(). Do you mean something else?

Oh yeah, you were using addColumn :thinking:

Regardless there were certainly some bugs… i wonder if there are more… later today ill try to recreate exactly what you tried (with addColumn and not adding a whole new header)

actually, it works much nicer with addColumn, no “flash” when you reuse it… http://haxeui.org/builder/?2f0d0297

I might see if i can get the same effect when recreating the entire header

Ok, that kind of solves the issue of being unable to get the width of columns wider than a few pixels - just don’t try customising the headers.

But is there another native-friendly way of doing per-column filters? I see sqlitebrowser appears to use the first row, but then I would guess there are other problems. Otherwise I guess they have to go outside the tableview…

Im not sure that there is… the native version uses wxWidgets and sqlitebrowser looks to use Qt… Qt allows MUCH more customization because its not actually native (it just emulates the native L&F very well), as such its not limited by the OS UI components. Wx on the other hand is 100% fully native which means you get alot less customization unfortunately.

You could maybe put the columns / filters in another scrollview above the table and sync the scrolling but a) i dont know how well that would actually work and b) im almost sure it will look / feel pretty bad…

I’ve removed the custom header components, and so now the test application can load CSV documents and display them in a rudimentary way. This is now commited on the main branch of the github repo linked above.

I’m wondering how large the CSVs can get before the application becomes unusable.

I’m finding the answer is: not very big! (This was trying with hashlink and linux targets using OpenFL.)

To illustrate this, modifying your test (here) to set the number of rows just one order of magnitude larger (100) makes it noticeably slower to create the table (ten seconds or so), and two orders makes it take minutes - and then the field content appears scrambled, suggesting something more than just slowness is happening. So I conclude it probably isn’t merely my implementation at fault (I was wondering if the CSV parser was slow, perhaps).

[edit: the scrambled content later disappeared, making me think HaxUI just wasn’t finished rendering yet]

This is a bit of a show-stopper, as I will definitely need to be able to view CSVs with thousands, probably tens of thousands of rows, and tens of columns. It’s also odd, since I’m pretty sure pure javascript in a browser could handle this size of table, and I’d have thought that hashlink / wxwidgets (which I infer is what the linux build uses) ought to be at least as fast as that.

Is there anything we can do to optimise that? For instance, I wonder how slow the reflection step needed to create the dataset is?

So for something with that many rows, you almost certainly want to use “virtualization”, which listview and tableview support (iirc, native components are already virtual), heres an updated example: http://haxeui.org/builder/?8cccf6e3

The only two changes here are:

  • i did table.virtual = true in the createDynamicTable function
  • i changed the number of items created from 10 to 100000

Holy crap, that makes a massive difference!

Are there any consequences of using this feature? Some sort of a trade off? I’m looking for some documentation but not finding any beyond it being listed in the TableView/ScrollView API documentation, and a mention of “Virtualisation” on the main HaxeUI page.

Yeah, documentation is a little sparse in general. The way it works is it reuses the item renderers and scrolls the data, rather than the scrollview contents.

So without virtual each item gets an item renderer, this is actually a number of components, so you can see how will a 1000s of rows you can end up up with 1000s upon 1000s of discrete components. With virtual you only get the the number of item renderers the viewport can show (+ an additional one for making scrolling look good), then when you scroll the table you actually scroll the data in the item renderers, and not the actual viewport.

There are trade-offs, the most obvious one being that the height of each item will be the same now, so if you have wrapped text it will simply get clipped: http://haxeui.org/builder/?f149778a