Are there any practical Pytorch examples? Say my network training time is 12 hours, I wonder how beneficial this would be for hyperparameter tuning over just simple grid/random search? Or would I instrument my network in a way to iterate over hyperparams faster than at every epoch/run?
We have not yet released examples of interfaces with Pytorch.
Maybe with moderate number of hyperparameters the benefit compared to random search will be moderate, whereas it will be very significant with high number of hyperparameters.
It also depends on how parallel you are. In all cases we have a wide range of algorithms with a common interface, so that you can compare.
We also use it for direct training of the weights of a network in reinforcement learning, not only hyperparameters.
A fundamental problem is as the number of parameters increase the probability of sampling from the edge of the hypercube increases. You will then not effectively explore the parameter space. This might be some what alleviated by a concentrated multivariate normal, but I guess that has its own caveat.
If you instead have a sampling algorithm informed by the loss functions you avoid this problem. (You instead might have to worry about local minima.)
For small numbers of hyperparameters, sometimes just random search is enough. This is not an absolute rule, sometimes with just 4 parameters random search miserably fails... just my rule of thumb, empirically, is that for hyperparameters in machine learning (this is certainly not the case in general) random search is often enough for 4 to 12 hyperparameters if the budget for hyperparameter search is ~100 trainings.
> ...there’s little hope of sharing and growing the world’s knowledge if those doing so ... cannot trust that their information will remain private.
Here's a crazy idea, circa 1990's: don't store their personal information! Allow people to browse Quora without using their real names. I'm very happy I deleted my Quora account when I did.
I just built a test site with Netlify + GitLab. Why must I give it full access to all of my GitLab repositories? I just want to deploy 1 repo. Seems like that opens up unnecessary possibilities for a security breach.
It's the only option with the current OAuth model, however, we only use the token generated in your web browser, and talk directly to GitLab's API from there. We use the token to add a deploy key and a webhook to the specific repository we're linking to, and after that it's discarded. That way there's no long lived token stored on our end that has access to your whole account.
I'd highly recommend folks create a tldr page for their CLI app. Add 4-8 examples to cover 80%+ of the most common use cases. -h flags, readmes & man pages can cover the other 20%.
I almost want to rewrite the help section to encourage examples even more. They're incredibly valuable.
I hadn't considered this before you mentioned it, but oclif CLIs could integrate to tldr pretty well. It already supports arrays of strings for examples.
Yup. On CPAN, it is encouraged that the first part of your documentation after the table of contents is the synopsis[1]. The synopsis should clearly show how to do the common tricks with the library. From there you can link and refer to the more detailed documentation.
We're doing that for our internal CLI applications and it's great to be able to just copy-paste the common use case from the top of the documentation without searching much.
I feel the synopsis section of man pages often just becomes a bunch of useless garbage above the fold (for instance, look at `man git`).
Using it less as a complete docopts kind of thing and more of multiple common usages (like `man tar` and what you linked) is far more useful.
I think there is something here I hadn't really considered before. It's not an example, but also not a useless dump of flags. Food for thought I suppose.
Python 3.7 is supported if you build master from source. But that can be a lot of work and though it may solve your TF problem, it may create other problems with other Python packages.
In my experience conda is the way to go for setting up your environment. I did not have great luck with pyenv, YMMV.
I don't know about WSL. Maybe setup a dual boot with Ubuntu on your system?
> The point will come when not regularly wearing one of these devices is considered hazardous to one's health
I 100% agree. I think we are on the cusp of the "cyborg" era, like it or not. Humans will wear health tracking devices at all times. Already 10M+ do, from basically 0 a decade ago.
Personally the sleep tracking and heart rate data from wearing my bands since 2014 led me to a diagnosis and fix for a number of health issues.
But the real promise lies ahead, as new and better sensors are engineered. In addition to helping with traditional health issues, devices will increasingly allow the conscious part of your mind to have real-time self-awareness of the biological factors going on that influence your thoughts and decisions and answer questions like these:
- Did you say "no" to that thing because it was the right decision or because your blood glucose level was so low?
- Was your high productivity level today a fluke or influenced by the high caffeine level in your system?
- How "tired" are you really?
Some people might not like it, but I don't know if they'll have a choice. Not taking advantage of these technologies would put them at a disadvantage in school, work, sports, and perhaps all other areas in life.
I think that moment is more like 10-20 years out, but otherwise agree with you.
I recently was looking to see where Samsung watch's health team was in the U.S. As far as I could tell there isn't one. I could be wrong, but that would seem to be a disadvantage compared to Apple/FitBit/Garmin/etc, since the more health focused these apps get the more they'll have to work with the FDA, insurers, hospitals, etc.
