commit for comments by rafael-hs · Pull Request #2 · rafael-hs/github-webscraping-elixir

rafael-hs · 2021-04-01T20:46:17Z

Valei-me

vivialima · 2021-05-20T20:55:05Z

lib/github_webscraping/extract_file_infos.ex

@@ -0,0 +1,63 @@
+defmodule GithubWebscraping.ExtractFileInfos do
+  def fetch_file_name(html) do
+    file_name =


não precisa atribuir a var se vc ja esta retornando o resultado da ultima linha da função

rfsbarreto

Já ouviu falar em teste irmao?

rfsbarreto · 2021-05-20T20:53:05Z

lib/github_webscraping/extract_file_infos.ex

+      |> Floki.text()
+
+    cond do
+      String.contains?(string_bytes, "KB") ->


EU prefiro usar pattern matching ao inves de conds. entao criaria uma funcao auxiliar pra lidar com essas diferencas de codigo ai

como eu iria fazer isso com pattern ? tem algum exemplo ?

def get_number_bytes(string_bytes, "MB") def get_number_bytes(string_bytes, "KB" ) def get_number_bytes(string_bytes, "GB" ) def get_number_bytes(string_bytes, "Bytes" )

E nessa linha ao inves de fazer o cond, vc faz:
get_number_bytes( string_bytes, String.contains?(string_bytes, "MB"))

Aproveitando, as linhas 1 e 2 de cada umas das conds sao iguais, ou seja, duplicidade de código que dá pra ser extraida pra funcoes novas :)

rfsbarreto · 2021-05-20T20:53:45Z

lib/github_webscraping/github_webscraping.ex

+  """
+  alias GithubWebscraping.MappingRepository
+
+  defdelegate get_repository_infos(url), to: MappingRepository, as: :process


DAve thomas dorme feliz

rfsbarreto · 2021-05-20T20:54:47Z

lib/github_webscraping/group_file_information.ex

+  end
+
+  defp get_all_lines(files) do
+    all_lines = Enum.reduce(files, 0, fn file, acc -> file.file_lines + acc end)


n precisa de duas linhas, nome do metodo já é suficiente pra entender

rfsbarreto · 2021-05-20T20:55:01Z

lib/github_webscraping/group_file_information.ex

+  end
+
+  defp get_all_bytes(files) do
+    all_bytes = Enum.reduce(files, 0, fn file, acc -> file.file_bytes + acc end)


mesma coisa do comentario acima

rfsbarreto · 2021-05-20T20:56:31Z

lib/github_webscraping/mapping_repository.ex

+    Enum.filter(urls, fn url -> url =~ "tree" end)
+  end
+
+  defp is_first_url(url) do


nome do metodo ta ruim.

first_url?(url) seria uma forma mais elixirzada, certo?

rfsbarreto · 2021-05-20T20:57:17Z

lib/github_webscraping/mapping_repository.ex

+    Enum.filter(urls, fn url -> url =~ "blob" end)
+  end
+
+  defp get_pastes_url(urls) do


get_directories_urls*
ALO CAMBLY

em minha defesa existe essa palavra em inglês

vivialima · 2021-05-20T21:00:11Z

lib/github_webscraping/extract_file_infos.ex

@@ -0,0 +1,63 @@
+defmodule GithubWebscraping.ExtractFileInfos do


Esse modulo recebe um arquivo html ne ? olhando as funções, talvez daria pra chamar só de File. HTMLFile.get_name

vivialima

Faltou testinhos, masss boaa 🚀

vivialima · 2021-05-20T21:44:09Z

lib/github_webscraping/github_webscraping.ex

@@ -0,0 +1,8 @@
+defmodule Lib.GithubWebscraping do


ficaram 2 arquivos com o mesmo nome. Talvez esse poderia ser só NomeDoProjeto.Api mesmo

vivialima · 2021-05-20T21:46:02Z

lib/github_webscraping/extract_file_infos.ex

+        bytes * 1_000_000.0
+
+      String.contains?(string_bytes, "Bytes") ->
+        bytes = String.trim(List.last(List.last(Regex.scan(~r/(\d+)/, string_bytes))))


Dava pra ter usado |>

vivialima · 2021-05-20T21:47:33Z

lib/github_webscraping/schemas/github_file.ex

+  @derive {Jason.Encoder, only: [:file_url, :file_name, :extension, :file_bytes, :file_lines]}
+  defstruct [:file_url, :file_name, :extension, :file_bytes, :file_lines]
+
+  def build(file_url, file_name, extension, file_bytes, file_lines) do


O elixir tem uma função struct(%GithubFile{}, [key: value]) que retorna um struct. Não precisaria do build

vivialima · 2021-05-20T21:50:22Z

lib/github_webscraping_web/controllers/github_webscraping_controller.ex

+
+  alias Lib.GithubWebscraping
+
+  def index(conn, %{"github_url" => github_url}) do


Essa rota seria index mesmo? ou show recebendo url? user_path GET /users/:id HelloWeb.UserController :show

https://hexdocs.pm/phoenix/routing.html

vivialima · 2021-05-20T21:56:56Z

lib/github_webscraping/mapping_repository.ex

+
+  @spec process(String.t()) :: map()
+  def process(url) do
+    files = mapping_repository_by_url(url)


vivialima · 2021-05-20T21:57:34Z

lib/github_webscraping/mapping_repository.ex

+  end
+
+  defp mapping_repository_by_url(url) do
+    urls = get_urls(is_first_url(url))


vivialima · 2021-05-20T21:59:36Z

lib/github_webscraping/mapping_repository.ex

+
+    other_files =
+      if Enum.count(pastes_url) > 0 do
+        Enum.map(pastes_url, fn url ->


💅 Enum.map(dirs, &mapping_repository_by_url/1)

vivialima · 2021-05-20T22:12:39Z

lib/github_webscraping/mapping_repository.ex

@@ -0,0 +1,84 @@
+defmodule GithubWebscraping.MappingRepository do


não sei se entendi o Mapping no nome. O que acha de algo como ExtractRepositoryInfo.run ...

ricardoebbers · 2021-05-20T22:17:39Z

lib/github_webscraping/extract_file_infos.ex

+      html
+      |> Floki.find("div.repository-content")
+      |> Floki.find("div.d-flex.flex-items-start.flex-shrink-0")
+      |> Floki.find("strong.final-path")
+      |> Floki.text()
+
+    Path.extname(file_name)


ricardoebbers · 2021-05-20T22:21:51Z

lib/github_webscraping/mapping_repository.ex

+    IO.puts("\n----------Returned files----------\n")
+    IO.inspect(files)
+    IO.puts("\n----------Returned urls-----------\n")
+    IO.inspect(pastes_url)


Eu n sei o quanto essa opinião vale num projeto Elixir, mas se eu estivesse fazendo isso em OO, criaria uma classe separada para lidar com "apresentação". Basicamente um Presenter, ou View.

Provavelmente faria um módulo separado para agrupar essa responsabilidade de output

eu só coloquei isso para testes mesmo

mas bom ponto de qualquer forma

commit for comments

a60126f

vivialima reviewed May 20, 2021

View reviewed changes

rfsbarreto suggested changes May 20, 2021

View reviewed changes

vivialima reviewed May 20, 2021

View reviewed changes

ricardoebbers reviewed May 20, 2021

View reviewed changes

		@@ -0,0 +1,63 @@
		defmodule GithubWebscraping.ExtractFileInfos do


		alias Lib.GithubWebscraping

		def index(conn, %{"github_url" => github_url}) do

		@@ -0,0 +1,84 @@
		defmodule GithubWebscraping.MappingRepository do

Conversation

rafael-hs commented Apr 1, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rfsbarreto left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vivialima left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants