Stream of Consciousness

Mark Eschbach's random writings on various topics.

Storing focused App Data

Categories: programming

Tags: golang timeseries osx

So AppWatcher is able to reliably capture the application I use which is fairly awesome. Turns out I switch applications a lot though. Suppression of seen apps has not yet yield the secret to which application is stealing focus. Time to push the data capture to the next level.

There are many options for embedded storage within Golang and OSX. Since AppWatcher effectively captures a stream of focused applications, time series might be an interesting choice. Giving nakabonne/tstorage a shot!

This will break the app into two parts:

  • Data capture writing the time series data. Initially thinking of using active for the metrics series with various things like bundleID as labels to the data.
  • Batch data analysis tool: queries the database. Initial results will just dump the active metrics.

Building the database

Using modules: go get -u github.com/nakabonne/tstorage. Added a new flag to store via tstorage with a data path.

Using a service style design the component will consume a channel and write a record. From a setup side we need to initial the data store like the follow:

type TStorageEngine struct {
	store tstorage.Storage
}

func NewTStorage(ctx context.Context, filePath string) (*TStorageEngine, error) {
	storage, err := tstorage.NewStorage(
		tstorage.WithTimestampPrecision(tstorage.Milliseconds),
		tstorage.WithDataPath(filePath),
	)
	if err != nil {
		return nil, err
	}
	return &TStorageEngine{store: storage}, nil
}

func (t *TStorageEngine) Close() {
    //TODO: Complain somewhere if there is a problem
    t.store.Close()
}

Seems to works like a charm. Sadly much of the API does not respect contexts so I will plumb them in up to the point of the calls.

Now for the meat and potatoes:

func (t *TStorageEngine) storeRecord(ctx context.Context, msg *appkit.RunningApplication) error {
	labels := []tstorage.Label{
		{Name: "bundleIdentifier", Value: msg.BundleIdentifier().Internalize()},
		{Name: "bundleURL", Value: msg.BundleURL().FileSystemPath()},
	}

	return t.store.InsertRows([]tstorage.Row{
		{
			Metric: "active",
			Labels: labels,
			DataPoint: tstorage.DataPoint{
				Timestamp: time.Now().Unix(),
			},
		},
	})
}

Have not figured out a good value so far. No context support either. Time to recall the data. Theoretically we have no idea what the labels are and we want all of the data. Something like this should work to just prove the query. From an API stand point having a third return value indicating no records or just leaving it nil would have been better in my opinion.


func (t *TStorageEngine) ReplayAll(ctx context.Context) ([]int, error) {
	_, err := t.store.Select("active", nil, 0, time.Now().Unix())
	if err != nil {
		if errors.Is(err, tstorage.ErrNoDataPoints) {
			return nil, nil
		}
		return nil, err
	}
	return nil, nil
}

No points recorded :-(. Looking at the created test directory I was running with there is a WAL with zero bytes. Perhaps this is from the lack of a value? Setting value to 1 still has a WAL of zero bytes and no other files. I would hope a WAL would survive an unclean shutdown. There are only 3 methods: Insert, Select, and Close on the returned interface.

Moving on to the early close problem I tried setting up handlers for SIGTERM and SIGINT via something like this:

	processContext, processDone := context.WithCancel(context.Background())
	defer processDone()
	procSignals := make(chan os.Signal, 10)
	go func() {
		for {
			select {
			case sig := <-procSignals:
				switch sig {
				case unix.SIGINT:
					fmt.Printf("SIGINT received.  Shutting down.\n")
					processDone()
				case unix.SIGTERM:
					fmt.Printf("SIGTEM received.  Shutting down.\n")
					processDone()
				default:
					fmt.Printf("Unknown signal received: %d, ignoring.\n", sig)
				}
			case <-processContext.Done():
				return
			}
		}
	}()

This causes NSRunLoop to exit immediately, not processing any events of interest. No values are returned from run and no reference to signals in the documentation which is interesting. Does the sigaction handler under the hood mean NSRunLoop has no input mechanisms? Perhaps the system uses signals under the hood and Go binds to all? Intercepting all signals does not produce anything.

So I am guessing there is another mechanism related to Mach and the NextStep system related to messaging, with the Golang runtime and the NextStep system fighting over signals. This mystery will need to wait for another time sadly.