Gonçalo Valério

Tag: Internet

  • Is it “/.well-known/”?

    Ironically, according to my experience, the .well-known directory doesn’t do justice to its name. Even in use cases that would fit nicely in its original purpose. 

    But I’m getting a bit ahead of myself. Let’s first start with what it is, then move to discuss where it’s used. But we’ll do this rapidly, otherwise this post will get boring really fast.

    Let’s look at what the RFC has to say:

    Some applications on the Web require the discovery of information about an origin before making a request.

    … designate a “well-known location” for data or services related to the origin overall, so that it can be easily located.

    … this memo reserves a path prefix in HTTP, HTTPS, WebSocket (WS), and Secure WebSocket (WSS) URIs for these “well-known locations”, “/.well-known/”. Future specifications that need to define a resource for such metadata can register their use to avoid collisions and minimise impingement upon origins’ URI space.

    So, briefly, it is a standard place or set of standard URIs, that can be used by people or automated processes to obtain (meta)data about resources of the domain in question. The purpose of the requests and the content of the responses doesn’t even need to be related to the web.

    The RFC introduces the need for this “place”, by providing the example of the “Robots Exclusion Protocol” (robots.txt), which is a good example… that paradoxically doesn’t use the well-known path.

    Now that the idea is more or less settled, here are other examples of cool and useful protocols that actually make use of it.

    ACME HTTP Challenge

    The use-case, here, is that an external entity needs to verify you own the domain. So to prove it, you place a unique/secret “token” in a certain path, in order for this entity to make a request and check that is true.

    Many Let’s Encrypt tools, make use of this approach.

    Security.txt

    This one is a bit obvious, and I already addressed it in previous posts (here and here). It is just a standard place to put your security contacts, so that researchers can easily find all the data they need to alert you about any of their findings.

    Web Key Directory (PGP)

    Traditionally, OpenPGP relied on “key servers” and the web of trust, for people to fetch the correct public keys for a given email address. With the “Web Key Directory”, domain owners can expose the correct and up-to-date public keys to associated addresses in a well-known path. Then, email clients can quickly fetch just by knowing the address itself.

    Lightning Address / LN URL Pay

    Sending on-chain Bitcoin to pay for a beer at the bar, or to send a small tip, is not that useful or practical at all (time and long addresses will get in the way).

    For small payments in Bitcoin, the lightning network is what you should use. While instantaneous, this approach requires a small dance between both wallets (showing QR code, etc.)

    Using a lightning address (which is essentially the same as an email address), solves this problem. You type the address and send the funds, done. Your wallet takes care of figuring the rest. To accomplish that, it fetches all the information from a standard place in the /.well-known/ path.

    I wrote about it before, and if you wish, you can buy me a beer by sending a few “sats” to my “email” address.

    • Suffix: lnurlp
    • Details 1 / 2

    Password-Change

    This feature allows password managers to know where the form to change the password of a given website is located. Allows users to go straight to that place from the password manager’s UI.

    Digital Asset Link

    Have you ever touched on a link while using your Android smartphone and received a suggestion to open it in a certain app instead of the predefined web browser?

    Me too, now you know how it is done.

    The whole list of recognized well-known URI’s can be found here. But I guess there are way more suffixes in use, since 2 of the 6 mentioned above are not there but are widely used within their ecosystems.

    That’s it, looking at the list above gives us a glimpse of how certain things are implemented and a few good ideas of things we could add to our domains/websites.

  • Improving your online privacy: An update

    Ten years ago, after it became clear to almost everyone that all our online activity was being tracked and stored, I wrote a blog post about simple steps a person could take to improve their privacy online.

    Essentially, it contains a few recommendations that everyone could follow to reduce their fingerprint without much effort. It wasn’t meant to be exhaustive, and it wasn’t meant to make you invisible online. If your personal situation needs more, you have a lot more ground to cover, which was totally out of the scope of that post.

    The target audience was the average Joe, that doesn’t like to be spied on. Specially by commercial companies that just want to show you ads, sell you stuff or use your habits against you.

    Many things have changed in the last 10 years, while others remained the same. With this in mind, I think it is time for an update to my suggestions, keeping in mind that no specialized knowledge should be required and the maximum amount of effort should not surpass 30 minutes.

    1. Pick an ethical browser

    For regular users on any computer or operating system, the main window to the outside world is the browser. Nowadays, this app is of the utmost importance.

    My initial suggestion remains valid these days, you should install and use Firefox.

    There are other browsers that could also do the trick, such as Brave or Safari, but my preference still goes to Mozilla’s browser.

    No matter your choice, you should avoid Chrome and Edge. If you want a more detailed comparison, you can check this website.

    Expected effort: 5 minutes

    2. Install important extensions

    Unfortunately, the default configuration of a good browser is not enough, even considering it already includes many protections enabled from the start.

    For a minimal setup, I consider 2 extensions indispensable:

    These will ensure that most spyware, included in a huge number of websites, isn’t loaded and does not leak your private information to third-parties. They will also block ads and other junk that make the web slow and waste your bandwidth.

    Expected effort: 2 minutes

    3. Opt out of any data collection

    This topic is specially problematic for Microsoft Windows users. However, it is becoming an increase prevalent practice in all software vendors.

    They will tell you they are collecting anonymous data to improve their products and services, while often the data is not that anonymous and/or the purposes are far wider than the ones they make you believe initially.

    Nowadays, Windows is an enormous data collection machine, so to minimize the damage, you should disable as much of this as possible. If this is your operating system, you can find a step-by-step tutorial of the main things to disable here (note: you should evaluate if the last 3 steps make sense for your case).

    If you use a different operating system, you should do a small research about what data the vendor collects.

    The next action is to do the same on your browser. In this case, in Firefox you should paste about:preferences#privacy in the URL bar, look for Firefox Data Collection and Use and then disable all options.

    Expected effort: 2–8 minutes

    4. Use a better DNS resolver

    This suggestion is a bit more technical, but important enough that I decided to include it in this guide that only covers the basics.

    With the new configuration that we set up on points 2 and 3, in theory, we are well protected against these forms of tracking. However, there are 2 big holes:

    • Are you sure the operating system settings are being respected?
    • Trackers on the browser are being blocked, but what about the other installed applications? Are they spying on you?

    To address the 2 points above, you can change your default DNS server to one that blocks any queries to sites tracking your activity. Two examples are Mullvad DNS and Next DNS, but there are others.

    Changing your DNS server can also help you block tracking on other devices you have less control, such as your phone or TV.

    The links contain detailed guides on how to proceed.

    Expected effort: 4–10 minutes

    5. Segregate your activity

    This step is more related to your behavior and browsing habits than to any tools that you need to install and configure.

    The goal here is to clean any data websites leave behind to track you across visits and websites through time.

    You should configure your browser to delete all cookies and website related data at the end of each session, and by this, I mean when you close your browser.

    In Firefox, you should again to about:preferences#privacy search for “Cookies and Site Data” and check the option: “Delete cookies and site data when Firefox is closed“.

    Sometimes this is impractical because it will force you to login into websites and apps all the time. A good compromise is to use “Multi-Account Containers“, they allow you to segregate your activity into multiple isolated containers, so you can limit any tracking capabilities.

    Expected effort: 3 minutes

    6. Prefer privacy preserving tools and services

    Most online services that common folk use, go to huge lengths to track your activities. For most of them, this is their business model.

    Luckily, there are drop-in replacements for common tools that will provide you with similar or better service:

    The above are just a few examples, these choices will depend on your own needs. At first, you might find them strange, but experience tells me that soon enough you will get used to them and discover they are superior in many ways.

    Expected effort: 3–5 minutes

    7. Adopt better habits

    I’m already a few minutes over budget, but hey, privacy is hard to achieve nowadays.

    For this last point, the lesson is that you must be careful with the information you share and make use of GDPR to control when someone is overstepping.

    Here are a few tips, just for you to get an idea:

    • Don’t provide your personal data just because they ask (input random data if you think it will not be necessary).
    • Always reject cookies and disable data collection when websites show those annoying pop-ups. Look for the “reject all” button, they usually hide it.
    • Even if websites don’t prompt you about privacy settings, go to your account preferences and disable all data collection.
    • Use fake profiles / identities.
    • When too much information is needed, and you don’t see the point, search for other alternatives.

    The main message is: Be cautious and strict with all the information you share online.

    Concluding

    If you followed up to this point, you already made some good progress. However, this is the bare minimum and I only covered what to do on your personal computer, even though some of these suggestions will also work on your other devices (phone, tablet, etc.).

    I avoided suggesting tools, services and practices that would imply monetary costs for the reader, but depending on your needs they might be necessary.

    Nowadays, it is very hard not to be followed around by a “thousand companies and other entities”, specially when we carry a tracking device in our pockets, attached to our wrists, or move around inside one of them.

    In case you want to dig deeper, there are a many sources online with more detailed guides on how to go a few steps further. As an example, you can check “Privacy Guides“.

    Now, to end my post with a question (so I could also learn something new), what would you recommend differently? Would you add, remove or replace any of these suggestions? Don’t forget about the 30-minute rule.

  • Managing Secrets With Vault

    I’ve been looking into this area, of how to handle and manage a large quantity of secrets and users, for quite a while (old post), because when an organization or infrastructure grow, the number of “secrets” required for authentication and authorization increase as well. Is at this stage that bad practices (that are no more than shortcuts) as reusing credentials, storing them in less appropriate ways or no longer invalidating those who are no longer in required, start becoming problematic.

    Yesterday at “Madeira Tech Meetup” I gave a brief introduction to this issue and explored ways to overcome it, which included a quick and basic explanation of Vault and demo about a common use case.

    You can find the slides of the presentation here and if you have any suggestion or something you would like to discuss about it, feel free to comment or reach through any of the contact mediums I provided.

  • Federated Tweets, or Toots

    Recently there was been a big fuss about “Mastodon“, an open-source project that is very similar to twitter. The biggest difference is that it is federated. So what it means?

    It means that it works like “email”, there are several providers (called instances) where you can create an account (you can setup your own server if you desire) and accounts from different providers can communicate with each other, instead of all information being in just one silo.

    Of course for someone that is in favor of an open web this is a really important “feature”.

    Another big plus is that the wheel wasn’t reinvented, this network is inter-operable with the existing “GNU Social” providers (uses the same protocol), so you can communicate and interact with people that have an account in an instance running that software. It can be seen like 2 providers of the same network running different software packages (one in PHP the other in Ruby) but talking the same language over the network.

    I haven’t tested it much yet, but given it is a push for a solution that is not centralized (which is a rare thing nowadays) and I think it is a small step in the right direction, So I’ve setup an individual instance for myself where I will publish regularly links of posts/articles/pages that I find interesting. Feel free to follow at https://s.ovalerio.net and if you know someone worth following in this network, let me know.

    Here are a few links with more information:

    List of instances where you can create an account

  • Managing a 100% remote company

    https://www.youtube.com/watch?v=e56PbkJdmZ8

    This video about Gitlab was posted recently and is a very interesting case-study on how a company can normally function while having all of its employees working remotely.

  • Log based analytics are still useful

    A long time ago, most of the modern website analytics software made the shift from relying on server logs to use client-side code snippets to gather information about the user, in this last category we can include as examples Google Analytics and Piwik. In fact, this paradigm allows to collect information with greater detail about the visitors of the website and gives developers more flexibility, however this can also be seen as the website owners imposing the execution of code on the user’s computing device that goes against his will and undermines his privacy (some people go as further as putting it in the same category as malware). Log based analytics software, last time i checked, is seen as a museum relic from the 90s and early 00s.

    However, as have been explained in a blog post named: Why “Ad Blockers” Are Also Changing the Game for SaaS and Web Developers and further discussed by the Hacker News community, we might need look again to the server-side approach, since the recent trends of using Ad blockers (which have all legitimacy, given the excesses of the industry) can be undermining the usefulness of the client-side method, given that most of the time the loading of the snippet and the extra requests that are required are being blocked. This is why server side analytics can be very handy again, allowing us to measure the “Ghost Traffic” as it is called in the article.

    A very high level overview of both methods can be described like this:

    Client-side:

    • Pros:
      • Lots of information
      • Easy to setup
    • Cons:
      • Extra requests and traffic
      • Can be blocked by browser extensions
      • The use of a third party entity raises some privacy concerns.

    Server-side:

    • Pros:
      • Cannot be blocked,
      • Does not pose a privacy concern since it only records the requests for the website “pages” made by the user.
    • Cons:
      • Less detailed information,
      • If the server is behind a CDN, not all requests will hit the server.

    The main issues with the use of log based tool is that they look ancient, some haven’t seem an update for a while and can take some work to setup. Nevertheless, they definitely can be very useful in order to understand the extent of the usage of blockers by visitors and even for the cases when we just need simple numbers. It also puts aside the privacy discussion since it only monitors the activity of the servers.

    That’s the case of this blog, I do not run any analytics software here (because I do not see the need given its purpose) and when I’m curious about the traffic, I use a very cool tool called GoAccess, that goes over the nginx logs and generates some nice reports.

    Give it a look, perhaps you don’t need Google Analytics everywhere or its results might not be as accurate as you think, specially if your audience has a significant percentage of tech-savvy people.

  • Managing secrets

    A few hours ago, I published a small article on Whitesmith’s blog about sharing and managing secrets, inside a software development environment. At first I dig a little into this problem that is very common and later I explain how we are addressing these issues. You can check it through the following link:

    Managing Secrets (www.whitesmith.co/blog/managing-secrets/)

    I mention some tools on the article that are very interesting in this area, but a more detailed analysis or walk-through was left for a future post as we get more familiarized with them.

  • Getting started with GPG

    Getting started with GPG

    Last week I gave a small workshop during the lunch hour (the famous Whitesmith’s “Lunch’n Learn”) about openPGP. It only covered the most basic aspects so that non-technical people could be introduced to this tool, something that can be very useful when someone is sharing or working with sensitive information (either personal or work related).

    It covers aspects such as key pair generation, key revocation, exportation/importation of keys and obviously encryption and decryption of documents. Most of the examples use a command line interface but there are several frontends available which are straightforward to use once you are familiarized with the concepts. Perhaps it can also be useful to you (somehow) so here is the link to the presentation, if you have any question feel free to ask in the comments section.

  • “Bloat”

    Last week I’ve read a great post entitled “Web Design: The First 100 Years“, it is a long one but definitely worth reading. I will just leave here a quote (3 short paragraphs) because it puts into words something that already crossed my mind multiple times.

    “A further symptom of our exponential hangover is bloat. As soon as a system shows signs of performance, developers will add enough abstraction to make it borderline unusable. Software forever remains at the limits of what people will put up with. Developers and designers together create overweight systems in hopes that the hardware will catch up in time and cover their mistakes.

    We complained for years that browsers couldn’t do layout and javascript consistently. As soon as that got fixed, we got busy writing libraries that reimplemented the browser within itself, only slower.

    It’s 2014, and consider one hot blogging site, Medium. On a late-model computer it takes me ten seconds for a Medium page (which is literally a formatted text file) to load and render. This experience was faster in the sixties.” Maciej Cegłowski

  • Now running over HTTPS

    2014 was a year where we witnessed a huge amount in Internet related security incidents, in the previous years, the world understood that our Internet usage is not only accessible to us and our service providers but to every middleman in the way as well. So it is natural that the recent trend is to secure our communications through the Internet to make the whole web safer, not only emails or chat conversations but our navigation too.

    Last year we saw the rise of many projects that pretend to improve the security of the Internet’s users. For example, most of the big companies enforced the use of HTTPS to access their services, CloudFlare’s launch of Universal SSL, the spawn of “Let’s Encrypt” project and major initiatives to promote the usage of private browsing tools like “tor” and encrypted chat apps like ChatSecure, TextSecure, etc.

    In the long run this will certainly contribute to better and safer web. That is why I decided to start this year with a major change on my personal websites and pet projects, for now on all of them will only be accessible through HTTPS. I know it is not a perfect system but it is a step forward.

    Full SSL
    Cloud flare Full SSL Scheme

    I am using Cloud Flare Full SSL, until the “Let’s encrypt” project takes off this summer. This approach however is not an end-to-end encryption scheme since all traffic is decrypted and encrypted again in the servers of the CDN. This is a drawback (and a deviation of the original concept of SSL) but it still is better than nothing and should be fixed by the summer.

  • The Web We Lost

    The Web We Lost, a two years old post with a message that still stands. For those who have an hour to spare, some good points are addressed in the video and in the Q&A afterwards.

  • Django Resources

    As I said in earlier posts in this blog, when i build websites or webapps where there are no technology impositions, i usually choose to do it in Python and in most of the cases, that’s the equivalent to say i choose to do it in Django.

    Over the last year, since i started using Bundlr,  I’ve been aggregating some resources like blog entries, tutorials and videos that i found useful and that could become handy in the future.

    Today I’m sharing the collection here, since it might helpful to someone else. I hope you like it and if you know more references that should be included in the list, please share it in the comments or send me an email.

    The list can be found here.

    Edit July 2016: Since I removed my account, the list is not longer available on Bundlr. Check recent posts, it will be published again soon.

  • Improving your online privacy

    Following this PRISM thing that’s going on for several weeks now, Internet privacy become a hot topic with extensive discussions and vast amounts of content being written about it (a good thing from my perspective).

    In this post I will try to sum some tips to improve your privacy and safety while surfing the web. The majority of this suggestions are also available in other websites, as they are the result from my searches about this subject and in some cases they might not be the best ones (if you can point some improvements in the comment area, i would appreciate).

    Before starting i just want to make clear that this isn’t aimed to make you invisible on the web (is it possible?) or to protect you from all threats since that is huge task and involves a great deal of technical knowledge (a good start would be here).

    Browser

    First of all, if we are talking about navigating through the Internet, the first step must be the web browser, because almost every tip that i will write on the following lines depends on it and is the application in which we spend the majority of our time.

    My choice goes to Firefox, because it’s not strongly tied to any company that makes most of their profit based on the ability of tracking you and it’s open-source software (although only a small percentage of the users ever looked at its source code).

    The other major reason is related with the posture of the non-profit organization that developed Firefox, and their public stand defending the open web and its freedoms.

    Add-ons

    • Ghosthery: This add-on lets you visualize every advertiser and tracker that is embeded on a webpage and enables you to block the unwanted codes.
    • HTTPS Everywhere: When possible this plug-in makes the browser use a secure connection.
    • Adblock Plus: Advertisements finance a big number of Internet sites, paying for the people that work on them, but sometimes they are used in an abusive form. This add-on blocks them and all of their tracking codes.
    • Collusion: Lets you visualize the vast network of entities that collected information about you during the time you were on-line.

    Preferences / Options

    On the “privacy” tab of your browser’s options/preferences, there are some features you can turn on that will help you keep the house clean. First you can start by selecting the “Tell site that i do not want to be tracked” option, i doubt it will be respected but it doesn’t take more than 1 min. The other features are the ones that could improve your privacy a bit, such as “Always use private browsing mode”. If you don’t want to bother with this, you can always set less restricting options such as when to clear cookies (my recommendation is “when the browser is closed”).

    Profiles and Accounts

    Disable / Remove accounts of services that you do not use anymore and old content. If it is not needed anymore there is no excuse to keep it online or to maintain open accounts in those web companies.

    Search Engine

    Like is stated in a video that i shared sometime ago, your search queries can be recorded, analyzed and combined with other data to build a profile about your habits and to give you the results they (or their servers) think you want, with the possibility of keeping important information away from you. To escape this bubble, one solution would be to use one search engine that would give you generic results and my suggestion is to use DuckDuckGo. Its results are fairly good and has some interesting features like !Bangs and instant answers.

    Behavior

    Sure, these things improve your ability to avoid being tracked, make you able to connect you a list of websites in a secure way and don’t record your searches linking them to your profile, but this is a small drop in a big ocean, since a great deal of the privacy protection depends on the user behavior and his choices while surfing the web. Some good pratices would be not to use your real name when it isn’t necessary, choose well your webservices provider giving attention to their privacy policy and their reputation (examples: webmail, file storage, instant messaging, etc.). and finally, the no-brainer, be careful about what you publicly write or share online.

    Note: If you want to go deeper, you can find a set of more holistic recommendations here.

  • Tracking and bubbles everywhere

  • Feito

    Generally when I have to develop a website or web application, I use technologies and frameworks that I am used to work with and I have more than basic knowledge about them, in most of the cases it turn out to be django and in some of them I also use node.js. With these two i can always achieve what I want without to much effort (with some exceptions).

    Some time ago I was asked to participate in a project that involved Ruby on Rails, a technology that I didn’t have too much knowledge and never had done anything with it, besides attending few workshops and talks about it (where I’ve just got a general idea how it worked). So it was time to give it a try and the fastest way to learn basics is to build something from the ground up with it and understand how stuff works along the way, and that’s what I’ve done at the time.

    So the first step was to find something or some idea that I wanted to work on and that would involve all the common techniques and stuff that you generally have to master when you start developing webapps.

    Here is what I’ve come up with:

    An inverted daily “To Do” list. So basically instead of making a list of what you have to do, you at the end of the day write what you have done and rank it with the amount of effort it took to do it. The system then store it, and shows to you in a pretty graph of your performance along some period of time. For a motivational boost at the end of each week it sends you an email with all tasks that you accomplished and the total amount of effort points.

    Basically it’s an app to monitor your daily performance and could serve as a motivational tool to help those who struggle in getting things done.

    I know its a basic app and I didn’t implement too many features, but it served its main purpose, at the time, of understanding the basics of ruby on rails.

    Yesterday I made it available on-line to anyone who wants to try it. For those who end up using it for some days/weeks I would appreciate some feedback, reporting of any errors you find or even suggestions of new features that would improve the app.

    So you can find it at: feito.ovalerio.net (Update: after 5 years, the server was turned off)